-- Logs begin at Sun 2022-07-31 13:20:19 UTC, end at Sun 2022-07-31 14:43:17 UTC. -- Jul 31 14:07:44.505341 dv-cinder-849846-4-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 31 14:07:44.512917 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: *** Starting uWSGI 2.0.18-debian (64bit) on [Sun Jul 31 14:07:44 2022] *** Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: nodename: dv-cinder-849846-4-iscsi Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: machine: x86_64 Jul 31 14:07:44.514937 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: clock source: unix Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: pcre jit disabled Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: detected number of CPU cores: 4 Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: current working directory: / Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: detected binary path: /usr/bin/uwsgi-core Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: your processes number limit is 47680 Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: your memory page size is 4096 bytes Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: detected max file descriptor number: 2048 Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: lock engine: pthread robust mutexes Jul 31 14:07:44.515261 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: thunder lock: enabled Jul 31 14:07:44.515737 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 31 14:07:44.515737 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 31 14:07:44.555801 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: Python main interpreter initialized at 0x5642faca99c0 Jul 31 14:07:44.555801 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: python threads support enabled Jul 31 14:07:44.555801 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: your server socket listen backlog is limited to 100 connections Jul 31 14:07:44.555801 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: your mercy for graceful operations on workers is 90 seconds Jul 31 14:07:44.556198 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: mapped 403077 bytes (393 KB) for 2 cores Jul 31 14:07:44.556271 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: *** Operational MODE: preforking *** Jul 31 14:07:44.556539 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: *** uWSGI is running in multiple interpreter mode *** Jul 31 14:07:44.556539 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: spawned uWSGI master process (pid: 80106) Jul 31 14:07:44.556745 dv-cinder-849846-4-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 31 14:07:44.558481 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: spawned uWSGI worker 1 (pid: 80120, cores: 1) Jul 31 14:07:44.558481 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: spawned uWSGI worker 2 (pid: 80121, cores: 1) Jul 31 14:07:44.558481 dv-cinder-849846-4-iscsi devstack@keystone.service[80106]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 31 14:07:50.094016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 14:07:50.094016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: from cryptography.utils import int_from_bytes Jul 31 14:07:50.094792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 14:07:50.094792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: from cryptography.utils import int_from_bytes Jul 31 14:07:50.095237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 14:07:50.095237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: from cryptography.utils import int_from_bytes Jul 31 14:07:50.095368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 31 14:07:50.095368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: from cryptography.utils import int_from_bytes Jul 31 14:07:50.143928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 14:07:50.144030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 14:07:50.144161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] command line args: [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 14:07:50.144295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 31 14:07:50.144386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 31 14:07:50.144461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 14:07:50.144579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] command line args: [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 31 14:07:50.144665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ================================================================================ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 14:07:50.144834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] admin_token = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.144929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 31 14:07:50.145002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ================================================================================ {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 31 14:07:50.145075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_dir = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] admin_token = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] config_source = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] config_dir = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] debug = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.145775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.146025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] config_source = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.146025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] debug = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.146139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.146207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.146294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_config_append = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_config_append = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.147979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_dir = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_dir = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_file = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.148769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.149996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.150238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.150315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.150431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.150510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.151981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] publish_errors = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] publish_errors = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.152948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] standard_threads = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] standard_threads = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.153936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_journal = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_json = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] use_journal = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_stderr = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] use_json = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] use_syslog = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] use_stderr = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] use_syslog = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.154956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.155120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 31 14:07:50.155202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.155970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.156914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.157855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.158947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.159927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.160992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.161897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.162979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.163951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.164947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.165866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.166016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.166094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.166304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.166911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.167872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.168942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.169947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.170889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.171994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.172945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.173950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.174882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.175836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.176932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.177921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.178999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.179978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.external = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.external = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.180900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.password = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] auth.token = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.password = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] auth.token = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.181971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.182895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.183861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.184982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.185943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.186960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.187999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.188943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.189919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.190845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.191964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.192978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.193919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.194889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.195984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.196926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.197996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.198918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.199869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.200958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.201975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.202887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.203862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.204934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.205943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.206941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.207984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.208928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.209866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.210944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.211988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.212935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.213606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.213691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.213845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] role.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] role.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.214853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.215843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.216965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.217924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.218868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.219876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.220957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.221981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.222901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.223879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.224966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.225981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.226896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.connection = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.227931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.228623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.229882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.230979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.231975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80120) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 14:07:50.232104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.232232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 31 14:07:50.232356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80121) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 31 14:07:50.260549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.260770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.260994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.261132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.261367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.261594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.269281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.270159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.299661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.299876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.300592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.300826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.352529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.352973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.403324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.403578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.404602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.404838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.468334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.468551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.468761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.468892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.469929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.492743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.492898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.493959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.494090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.494218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.494320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.495012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.505567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.505725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.518356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.518588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.518714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.518866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.518995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.519913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80120) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.520038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.520038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80121) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 31 14:07:50.536463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.536614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.536742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 14:07:50.536868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 14:07:50.536993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.537130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.537256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 14:07:50.537382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 31 14:07:50.537506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.537632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.537764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.537900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.538025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.538150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.538275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.538400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.548304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.548575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.548840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.549022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.549149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.549293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.549546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.549823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.555993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.556732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.561184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.561461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.561738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.561938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.562065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.562891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.563960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.564089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.564315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.564576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.587328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.587971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.588866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.589142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.589485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.589761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.593717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.593999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.594278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.594415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.594629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.594904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.600823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.601100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.601466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.601745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.605983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.606228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.606355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.606512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.606639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.606777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.606907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.607928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.608009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.608222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.624370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.624652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.624786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.625057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.628583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.629008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.631711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.632000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.632148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.632438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.636398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.636844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.639573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.640053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.642767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.643949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.644116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.644231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.644387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.644677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.660142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.660422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.660636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.660762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.660916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.661055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.661182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.661432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.668312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.668883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.669844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.670114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.670413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.670685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.672943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.673219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.673517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.673786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.673912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.674065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.674191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.674360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.674634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.674916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.686001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.686276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.686549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.686822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.686957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.687083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.687222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.687350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.687517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.687645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.687769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.688003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.688272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.688544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.703341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.704430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.706322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.707385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.709252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.710340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.711711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.712780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.713221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.713496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.713772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.714396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.714670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.714945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.720282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.721397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.721854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.722976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.724787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.725078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.725354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.725633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.725918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.726213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.726495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.726774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.734142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.734426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.734704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.735307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.735613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.735891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.741532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80120) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.741808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.742085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.742356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.742627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.742762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80121) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 31 14:07:50.742895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.743873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.744013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.744141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.744855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80120) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.745736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.746017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.746277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.746548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.746805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80121) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 31 14:07:50.778722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642faca99c0 pid: 80120 (default app) Jul 31 14:07:50.781576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5642faca99c0 pid: 80121 (default app) Jul 31 14:07:50.781979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be201e-6fe1-4814-8f82-a9a4317c9d6a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:07:50.782265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be201e-6fe1-4814-8f82-a9a4317c9d6a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:07:50.782536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be201e-6fe1-4814-8f82-a9a4317c9d6a None None] PATH_INFO: `/v3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:07:50.786783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Sun Jul 31 14:07:50 2022] GET /identity/v3/ => generated 255 bytes in 7 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:07:58.350544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:07:58.351112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:07:58.351112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:07:58.355482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:07:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:07:58.360690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:07:58.361097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:07:58.361236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:07:58.430873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: INFO dbcounter [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] Registered counter for database keystone Jul 31 14:07:58.450832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG oslo_db.sqlalchemy.engines [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80120) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 14:07:58.456889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writer thread running {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 14:07:58.630630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG passlib.handlers.bcrypt [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80120) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 14:07:58.631053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG passlib.handlers.bcrypt [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80120) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 14:07:58.686880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:07:58.688433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:07:58.832396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:07:58 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 473 msecs (HTTP/1.1 201) 6 headers in 384 bytes (2 switches on core 0) Jul 31 14:07:58.957048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:07:59.026471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: INFO dbcounter [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] Registered counter for database keystone Jul 31 14:07:59.047674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG oslo_db.sqlalchemy.engines [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80121) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 31 14:07:59.053740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writer thread running {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 31 14:07:59.171810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb984c-4df2-4382-a027-5c0f1043fcf2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:07:59.176051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:07:59.176690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:07:59.176969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:07:59.177241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:07:59.185206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:07:59.237133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING keystone.common.rbac_enforcer.enforcer [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 14:07:59.237416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4a9539-20c2-4b65-a216-c9e53c1a352e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:07:59.246848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Jul 31 14:07:58 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 290 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:07:59.252902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:07:59.266948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c429c1-fe87-4912-a416-69c33194f596 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:07:59.271235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:07:59.272390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:07:59.272673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:07:59.272946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] PATH_INFO: `/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:07:59.274474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] RBAC: Authorizing `identity:get_project(project_id=0e21ca24dfa94435a41c5ab2bd838cb9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:07:59.325694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 31 14:07:59.326030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ed5e04-8841-49eb-aacf-36b377de1ae5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:07:59.330528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Sun Jul 31 14:07:59 2022] GET /identity/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9 => generated 344 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:01.348386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6766f-154c-4e44-9a05-cc8a6b6ec860 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:01.348917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6766f-154c-4e44-9a05-cc8a6b6ec860 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:01.348917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a6766f-154c-4e44-9a05-cc8a6b6ec860 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:01.353454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:01 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:01.358710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:01.359135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:01.359271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:01.379370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:01.380931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:01.389433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 31 14:08:01.410631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6766f-154c-4e44-9a05-cc8a6b6ec860 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:01.422499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a6766f-154c-4e44-9a05-cc8a6b6ec860 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:01.426672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:01.427272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:01.427587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:01.427865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:01.435970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:01.437728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7274d1dc-2136-46ac-ac55-4024eaf7fe11 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:01.446326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Sun Jul 31 14:08:01 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:08:01.452376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:01.461209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b1200c-9619-4c96-a7ee-9439a38c11c3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:01.465622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:01.466248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:01.466662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:01.466944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:01.468334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:01.470148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852c025b-4618-4104-8797-d9cb4ccdada2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:01.471723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Sun Jul 31 14:08:01 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:03.527149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f184ee-ada1-4442-ad36-7a8d0cfba7e8 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.527659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f184ee-ada1-4442-ad36-7a8d0cfba7e8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.527659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f184ee-ada1-4442-ad36-7a8d0cfba7e8 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.528998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:03.534138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83677-c43c-4d31-9559-aed7cc17615e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.534585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83677-c43c-4d31-9559-aed7cc17615e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.534744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c83677-c43c-4d31-9559-aed7cc17615e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.554114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b5c83677-c43c-4d31-9559-aed7cc17615e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:03.555322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b5c83677-c43c-4d31-9559-aed7cc17615e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:03.635002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 6/11] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:03.647882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b115-f33d-4455-a951-c94c06197e1e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.648166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b115-f33d-4455-a951-c94c06197e1e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.648446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1370b115-f33d-4455-a951-c94c06197e1e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.650288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 6/12] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:03.655368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.655683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.655967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.658364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b115-f33d-4455-a951-c94c06197e1e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:03.674180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:03.675388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:03.691133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:03.746672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:03.768051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:03.803628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1370b115-f33d-4455-a951-c94c06197e1e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:03.836872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:03.838601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.838889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.839175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.852245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:03.862864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:03.864179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:03.865239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:03.870646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:03.879586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914b9c0d-786a-4113-b4a3-a95673b3265d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:03.903501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:03.913375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:03.914014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:03.914300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:03.914579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:03.936164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:03.975480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe9f798-fe7a-4225-be14-4e1493483e17 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:03.982691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:03.984312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.007383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.008023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.008310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.008869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.010089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:04.010393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.011795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.023124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:04.029363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:04.081585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:08:04.086920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:04.120798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:04.130374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cfebb2-a15a-4ab8-8bd2-d24699d92472 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:04.138686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:04.169786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.170439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.170716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.170984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.172599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.174040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.175660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:04 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:04.181457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:04.187337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0447606-4002-4968-82d2-1c2c7a473123 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:04.222093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.222735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.223017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.223294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.233094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.234547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.280928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34426eba-b370-47bd-b268-ecc90117c492 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:04.315703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.316368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.316652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.316932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.318246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.319770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.321897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:04.327942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:08:04.358445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:04.359668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:04.360678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:08:04.360678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:08:04.360678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:08:04.360678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:04.360678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:04.361577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:04.365809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:04.431884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c4ab2-02f2-4e5c-8f70-c7f33865e242 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:04.464629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.465251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.465535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.465625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772f6954-2718-4345-86cf-22e4007a40a1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:04.465814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.466606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.domain.id=default, user.id=da5b1c90e892464380eacee9a29f0449) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:04.466921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.467947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:04.468623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8f8d0-4aa1-45e3-8caa-ca1f72317edb None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.476932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Jul 31 14:08:04 2022] GET /identity/v3/role_assignments?user.id=da5b1c90e892464380eacee9a29f0449&scope.domain.id=default&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 233 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:04.501831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:04.502473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:04.502760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:04.503034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:04.503791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:04.504102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:04.505530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682982ee-4941-4938-8543-30a61c89cc4a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:04.525032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:08:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:06.621602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79045e-9f1d-4746-a8a9-5198a9bf3445 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:06.622495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79045e-9f1d-4746-a8a9-5198a9bf3445 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:06.622950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d79045e-9f1d-4746-a8a9-5198a9bf3445 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:06.624504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:06.630091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:06.630507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:06.630659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:06.650321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:06.651544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:06.724941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d20d-36b6-459b-aabc-ec49fa6ca1f3 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:06.725280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d20d-36b6-459b-aabc-ec49fa6ca1f3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:06.725500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b65d20d-36b6-459b-aabc-ec49fa6ca1f3 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:06.727067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:06.736750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:06.737047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:06.737329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:06.739541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:06.773837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:06.775286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG passlib.handlers.bcrypt [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80121) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 31 14:08:06.775767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG passlib.handlers.bcrypt [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80121) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 31 14:08:06.848507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:06.850112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:06.894757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:06.903329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1c5fc8-a165-4d8a-a3f7-e540c62910da None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:06.939357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:06.940030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:06.940311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:06.940587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:06.953120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:06.960014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:06.960869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:06.962573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 12/25] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:06.969441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.032988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 14/26] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:07.068194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.147973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:07.150221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a3a2-b0d7-459f-8ead-d2408302f1b7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.248956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.249828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.250275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.250705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.251597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:07.252059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.253588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.278690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 13/27] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:07.288383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.296370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d6d260-7345-4954-8ba1-7d0ad3ec59eb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.332631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:07.373086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.374005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.374292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.374570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.472036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4a21d3-2b7f-4e75-8e9a-3a5c619e940b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.513694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.515230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.528961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.529623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.529906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.530188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.536154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.537640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.539274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 14/28] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:07 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:08:07.542381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:07.543175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:07.543991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:08:07.543991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:08:07.543991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:08:07.543991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:07.548783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.551704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 15/29] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 484 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:07.557982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.711828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:07.715910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:07.739653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59311510-8764-4c65-b048-c9a7ee60e92a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.739836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5894d3-71ad-48dd-a491-3486aff583e0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.777291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.777926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.778212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.778492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.779203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:07.779540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.781165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-830c15a2-0afe-4177-8203-aca27b22d4c2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.791964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.792621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.792904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.793183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.794351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.795821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f196e7-91d3-41a0-8ac0-ee2fa2349fb4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.797736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 15/30] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:07.798941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226592-9424-426e-89ca-bf636db9e98c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.799226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226592-9424-426e-89ca-bf636db9e98c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.799525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e226592-9424-426e-89ca-bf636db9e98c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.800027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 16/31] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:08:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:07.801234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 16/32] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:07.802909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e226592-9424-426e-89ca-bf636db9e98c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.807688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20a02-d86a-4b63-b0aa-25ad5825836f None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.807999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20a02-d86a-4b63-b0aa-25ad5825836f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.808285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc20a02-d86a-4b63-b0aa-25ad5825836f None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.809800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 17/33] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:07 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:07.810330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc20a02-d86a-4b63-b0aa-25ad5825836f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:07.909557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e226592-9424-426e-89ca-bf636db9e98c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.934921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc20a02-d86a-4b63-b0aa-25ad5825836f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:07.944674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.945354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.945642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.945924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] PATH_INFO: `/v3/domains/default/users/da5b1c90e892464380eacee9a29f0449/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.949070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=da5b1c90e892464380eacee9a29f0449, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.950941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561de094-445c-4319-8ca9-e8a878880bd6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.976171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:07.976999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:07.977365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:07.978242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:07.979288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:07.980929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50f71ac-0141-4187-8e39-e2d432a9811b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:07.992846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.083075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 17/34] 10.222.0.40 () {58 vars in 1361 bytes} [Sun Jul 31 14:08:07 2022] PUT /identity/v3/domains/default/users/da5b1c90e892464380eacee9a29f0449/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:08.085010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.085385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.085761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.107572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:08.109119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:08.118196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.136864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 18/35] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:08:07 2022] POST /identity/v3/projects => generated 301 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:08.138577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.138875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.139162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.184935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:08.186787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:08.241278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 18/36] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:08.277543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:08.294916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 19/37] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:08.375243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:08.426164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e29ac9e-8080-4332-a57f-a5ff70cfe350 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:08.464076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:08.464973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.465406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.465830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.466718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:08.468502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5893ea1a-f287-4fa9-9234-a6f120645b71 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:08.495910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 19/38] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/roles => generated 225 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:08.497755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a060b9-06fe-4044-8d81-46bf5135a314 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.498225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a060b9-06fe-4044-8d81-46bf5135a314 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.498685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a060b9-06fe-4044-8d81-46bf5135a314 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.500217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 20/39] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:08.509315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.509894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.510356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.516285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb20938-8285-483e-9c31-f042b33ded43 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:08.524114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.544506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.547956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:08.552024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:08.564678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:08.565806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.566326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.566931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.568070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:08.570020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d048f6-4952-406e-a0f8-d22c3dba296d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:08.599501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 20/40] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/roles => generated 219 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:08.601314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9101-6e49-452e-8831-b739900fbe5c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.602059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9101-6e49-452e-8831-b739900fbe5c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.603213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dc9101-6e49-452e-8831-b739900fbe5c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.605165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 21/41] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:08.613272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.614599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.615882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.659542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:08.660773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:08.667287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 21/42] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:08.690269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:08.769161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 22/43] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:08.797503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:08.820105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7946ff2a-8125-4a1a-a54c-16fff96f4974 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:08.859733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.864070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:08.864884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.865147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.865466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.866519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:08.868573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:08.887142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 22/44] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/roles => generated 223 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:08:08.928382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c045d53-2ed2-4a14-a4c4-a0dd06ec40dd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:08.941994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:08.974005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:08.974794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:08.975085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:08.975363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:08.976689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:08.978758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cceabc-8445-4b81-bca2-58035238dfa9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:08.980776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 23/45] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:08.986443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:09.095403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a01502b-b1dd-457a-95f8-0eebb461e475 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:09.130654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:09.131314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:09.131639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:09.131922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:09.132629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:09.134100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f984ae7-c4ae-4e99-9e22-39147eaa8d36 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:09.162626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 23/46] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:08:08 2022] POST /identity/v3/projects => generated 312 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:08:10.380834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71951315-7756-4b60-9613-58c7e84d8889 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:10.381340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71951315-7756-4b60-9613-58c7e84d8889 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:10.381442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71951315-7756-4b60-9613-58c7e84d8889 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:10.382814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 24/47] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:10.388729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:10.389167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:10.389287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:10.406733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:10.407885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:10.447361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:10.473006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 24/48] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:10.495704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71951315-7756-4b60-9613-58c7e84d8889 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:10.593993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71951315-7756-4b60-9613-58c7e84d8889 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:10.626504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:10.627145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:10.627449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:10.627746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:10.638118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:10.643113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:10.644090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:10.644946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 25/49] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:10.650124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:10.745710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c924193-03a9-49aa-9de6-11f4b113f05b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:10.778649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:10.779279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:10.779620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:10.779900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:10.780628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:10.780936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:10.782361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:10.791033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 25/50] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:10.796468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:10.836143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:10.894674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbc885-5a6d-411e-bf1b-ac4040a3c204 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:10.927382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:10.929098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:10.929454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:10.929738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:10.931728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:10.933184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:10.935010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 26/51] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:10 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:10.940269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:10.973923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:11.036826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759ae9e-217f-4d27-b6d9-17148b97dcea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:11.069563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:11.070204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.070489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.070767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.072213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:11.073707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac88f97-55e2-4243-a55b-9784a5255b28 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:11.075306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 26/52] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:11.081030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:11.154826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822d9a-e178-4785-a02e-b2330538b248 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.155114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822d9a-e178-4785-a02e-b2330538b248 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.155396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef822d9a-e178-4785-a02e-b2330538b248 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.156797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 27/53] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:11.161641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.161927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.162216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.180147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:11.181378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:11.184499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e018d753-ffe2-4405-9f51-f613153b7530 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:11.191267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:11.218384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:11.218997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.219280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.219589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.220390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.domain.id=default, user.id=da5b1c90e892464380eacee9a29f0449) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:11.220705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:11.222165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:11.228456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 27/54] 10.222.0.40 () {56 vars in 1322 bytes} [Sun Jul 31 14:08:11 2022] GET /identity/v3/role_assignments?user.id=da5b1c90e892464380eacee9a29f0449&scope.domain.id=default&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 530 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:11.254808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 28/55] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:11.278491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:11.382627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae2b20b-9a9c-410f-8de1-e92fbe365b60 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:11.417494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:11.418140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.418424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.418708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.419906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:11.421388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f59c21c-bafb-4a5d-99c7-67dfa1fce93c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:11.422984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 28/56] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:11.428617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:11.462710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:11.525742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f6910d5-aa02-4e30-94bc-b013721fde3d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:11.557051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:11.557660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:11.557938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:11.558209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:11.558900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:11.560369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26155843-fdf2-492f-b726-56040d939ca1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:11.585481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 29/57] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Jul 31 14:08:11 2022] POST /identity/v3/projects => generated 298 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:13.595931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657d26-c666-4d97-8880-92061a76628c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:13.596422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657d26-c666-4d97-8880-92061a76628c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:13.596422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de657d26-c666-4d97-8880-92061a76628c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:13.597768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 29/58] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:13.602907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:13.603280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:13.603490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:13.621436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:13.622598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:13.685187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 30/59] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:13.708846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657d26-c666-4d97-8880-92061a76628c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:13.750537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:13.810719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de657d26-c666-4d97-8880-92061a76628c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:13.842747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:13.843376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:13.843682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:13.843956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:13.845101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:13.846573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc524b-ebc5-4094-99dc-480cfb51b17b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:13.848135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 30/60] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:13.854163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:13.871355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:13.949695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da132cf-c409-43f8-98f1-04dc3517620e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:13.982056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:13.982661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:13.982940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:13.983214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:13.983920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:13.985357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4cd800-c9f3-43ac-8923-a635fd03d0e5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:14.010378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 31/61] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:13 2022] POST /identity/v3/users => generated 275 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:17.185895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05081f87-3a36-4b33-bf5c-fbcf7f31474c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.186543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05081f87-3a36-4b33-bf5c-fbcf7f31474c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.186543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05081f87-3a36-4b33-bf5c-fbcf7f31474c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.187871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 31/62] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:17.195912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.196701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.196701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.244722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:17.246164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:17.247367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd955d3-5052-4d9d-92da-647192e5838e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.247941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd955d3-5052-4d9d-92da-647192e5838e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.249454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbd955d3-5052-4d9d-92da-647192e5838e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.251051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 32/63] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:17.256343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588adc9-409e-4330-b984-0468e31eafd0 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.256878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588adc9-409e-4330-b984-0468e31eafd0 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.258405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8588adc9-409e-4330-b984-0468e31eafd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.290629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8588adc9-409e-4330-b984-0468e31eafd0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:17.292539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8588adc9-409e-4330-b984-0468e31eafd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:17.367794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 32/64] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:17.392965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 33/65] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:17.398194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.401173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.402947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c02b-b2eb-4535-976c-297a530f55bc None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.403239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c02b-b2eb-4535-976c-297a530f55bc None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.407707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5c02b-b2eb-4535-976c-297a530f55bc None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.409100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 34/66] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:17 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:17.416725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.417203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.417652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.435839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:17.437301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:17.495944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.505215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 35/67] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:17.506578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.513560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cad1bb-c605-4390-b883-a2706e556b67 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.569052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.569963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.570389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.571654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.582554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:17.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:17.589288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:17.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:17.592718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 33/68] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:17.593781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.624163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef08e828-2acc-4c51-8fd7-5039d4496e2d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.660319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.661615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.662310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.662782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.663064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.675749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:17.681245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:17.682100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:17.682854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 36/69] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:17.684209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.709510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab16731a-e2ff-4bed-a1be-965c36e7825c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.713008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.746572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.747330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.747907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.748138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.760163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-97525a83-2455-4d78-a903-b9e2922c1504 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:17.765230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:17.766093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:17.766843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 34/70] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:17.768269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b513e-81d3-4d88-833c-c354042aa1f2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.768269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b513e-81d3-4d88-833c-c354042aa1f2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.768269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460b513e-81d3-4d88-833c-c354042aa1f2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.769620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 35/71] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:17 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:17.774561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b513e-81d3-4d88-833c-c354042aa1f2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.803089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c89f23-7d57-4585-82b1-f4593ac78fea None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.846129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.846751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.847032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.847310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.848065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:17.848463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:17.849848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:17.858211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 37/72] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:17.858994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:17.875690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460b513e-81d3-4d88-833c-c354042aa1f2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.898021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.909934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.910604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.910838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.911114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.911869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:17.912173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:17.913558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408f7065-1359-4617-8024-d4b209b086f2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:17.921337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 36/73] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:17.922464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.922752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.923030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.941166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:17.942344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:17.944007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:17.959445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32647c69-d650-4059-98ae-e5e00b204e9a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:17.993125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:17.993746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:17.994023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:17.994296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:17.995009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:17.995309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:17.996707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4deb74-41d7-480f-8182-f3af33ac7711 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.004757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 38/74] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:18.005881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b572c-5dd9-4fcf-a2e0-05ef89540b47 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.006173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b572c-5dd9-4fcf-a2e0-05ef89540b47 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.006452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d51b572c-5dd9-4fcf-a2e0-05ef89540b47 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.007686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 39/75] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:18.009134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b572c-5dd9-4fcf-a2e0-05ef89540b47 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.015219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 37/76] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:18.018997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.113048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d51b572c-5dd9-4fcf-a2e0-05ef89540b47 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.118571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb7a6e4-ad2a-4e93-a62f-a9cee33c3a2b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.132552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.150219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.150859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.151138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.151410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.152595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.154026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.155634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 40/77] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:18.155987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.156642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.156928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.157205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.160813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.170622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.172110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64757164-29c0-4587-ab27-b4674aaeb219 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.173757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 38/78] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:18.174898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.175186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.175484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.179599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.193732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:18.194897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:18.258997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b44c92-9429-41e5-939f-20ff32be82b5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.259835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 39/79] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:18.260738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.296359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.297002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.297279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.297548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.298710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.300169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.301765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 41/80] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:18.302517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.360863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7a01fc-4511-4412-9024-e9b95be78a96 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.362329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.386149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.394772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.395408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.395715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.395989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.397143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.398620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.400219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 40/81] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:18.400974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.401350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b7ce72-1099-4721-8737-fe8f0ceb61c9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.435142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.435720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.436007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.436288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.500238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37818b2-07c7-4e25-bcdb-e02debb6c8b6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.501482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.502958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.533123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.533741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.534027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.534306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.578003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.588274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.606192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 42/82] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:18.607034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.610783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.612300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.704214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab863898-9a4d-4fc4-8091-adf5e195a34c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.713447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 41/83] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:18.714374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.741140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.741770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.742047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.742347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.752125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:18.756931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:18.757765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:18.758569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 43/84] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:18.759196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:18.786227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.808021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:18.817274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0990a5f0-04c7-4776-89a0-8ab70d453c6e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.850908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.852420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.852705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.853023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.861432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68742e7d-b50f-4a62-a489-eb6b56e6b6aa None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:18.902848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:18.903491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:18.903772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:18.904043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:18.904728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.906175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.921920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:18.923380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:18.932762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 44/85] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:18 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:18.933576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.011972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.046974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399a083-55c8-431b-8f5d-cd4d92b880d1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.049091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.055131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 42/86] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:19.055131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.096956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.096956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.096956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.096956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.098419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.100448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.101769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 45/87] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:18 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.103269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.184604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a2b329-79d1-4bd0-b851-a182c9b17d24 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.234428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f094e12-a6bb-451f-8bf7-35757ac7cc78 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.243430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.243430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.243430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.243430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.245242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.246095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.248098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 43/88] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.248877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.293331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.293878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.294152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.294428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.295162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:19.295495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.296883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.307282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.310446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 46/89] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.312200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.354349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.371221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bb50d-1f23-4d86-b46e-702969df1161 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.406479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.407237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.408212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.408212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.409172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.410724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.412990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 44/90] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.413904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.448073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8489f4-9ff3-410f-97c4-d472f6b270e5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.491452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.493380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.493981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.494528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.495522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=da5b1c90e892464380eacee9a29f0449) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:19.496137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.498434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.505733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 47/91] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/role_assignments?user.id=da5b1c90e892464380eacee9a29f0449&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.506776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.543197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f5b72-d80b-426d-9415-4e16ec066832 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.561702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.632059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d451eac9-9dea-4896-9bec-e23b55f5551d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.634836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:19.671793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.672854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.673425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.673873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.675273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.676999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.678786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 48/92] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.679681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.680596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.680873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.681158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.681506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.682299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:19.682613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.684145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.699012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 45/93] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 259 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:19.699896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:19.849535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a352b062-32d4-49d0-bc94-6029f339cfb8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:19.908651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:19.909638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:19.910088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:19.910654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:19.911665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:19.912174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:19.914001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:19.921396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 49/94] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=cce34632ed81427db845cc21f77efac0 => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:20.035361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b4f867-573a-4649-ab14-a6a66fe33efe None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:20.077812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:20.078790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:20.079245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:20.079712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:20.149417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:20.190928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:20.192464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:20.314730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 46/95] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 615 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:20.320074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:20.359860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:20.466009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1629e4f5-508a-4b0c-96ec-7676daa9f32b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:20.509371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:20.510023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:20.510306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:20.510585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] PATH_INFO: `/v3/projects/02d404154c9f45158242839af9898f0e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:20.514900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] RBAC: Authorizing `identity:get_project(project_id=02d404154c9f45158242839af9898f0e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:20.516396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf71ede9-6d61-482e-90d1-fdfaaabfcc06 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:20.518377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 50/96] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:20 2022] GET /identity/v3/projects/02d404154c9f45158242839af9898f0e => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:20.525004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:20.614034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:20.653055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58dc863-b622-4ff6-84fd-078d3ceae6ed None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:20.688579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:20.689208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:20.689496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:20.689780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:20.690584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=02d404154c9f45158242839af9898f0e, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:20.690897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:20.692371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed94f7ac-d215-4d35-b4db-f836b365739d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:20.698947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 47/97] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:20 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=02d404154c9f45158242839af9898f0e&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:21.562622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38946-5f21-419b-bb4d-86a406f26a60 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:21.563098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38946-5f21-419b-bb4d-86a406f26a60 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:21.563098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae38946-5f21-419b-bb4d-86a406f26a60 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:21.564793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 51/98] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:21.570150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:21.571017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:21.571778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:21.595611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:21.598492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:21.682820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 48/99] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:21.719981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae38946-5f21-419b-bb4d-86a406f26a60 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:21.858477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:21.872075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae38946-5f21-419b-bb4d-86a406f26a60 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:21.913954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:21.915136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:21.915451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:21.915724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:21.932274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:21.937374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:21.938215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:21.938952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 52/100] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:21.947067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.007498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:22.060800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a1a04c1-2660-4106-84cb-72fd2be2b3a0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:22.100165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:22.100797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.101076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.101360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.102122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:22.102427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:22.103849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e15702b-9143-49af-ba79-ef301c8c1c65 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:22.112661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 49/101] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:22.118081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.156107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea683-e7c0-4b4c-8245-c8169cfca19c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.156573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea683-e7c0-4b4c-8245-c8169cfca19c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.157015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62ea683-e7c0-4b4c-8245-c8169cfca19c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.158513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 50/102] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:22.163740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23200df6-0f38-49e4-9fa9-cf32e9a373ff None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.164727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23200df6-0f38-49e4-9fa9-cf32e9a373ff None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.165165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23200df6-0f38-49e4-9fa9-cf32e9a373ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.190018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-23200df6-0f38-49e4-9fa9-cf32e9a373ff None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:22.192026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-23200df6-0f38-49e4-9fa9-cf32e9a373ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:22.289826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 51/103] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:22.291320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3016-31b3-4853-a7fe-9ac299f2c555 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.291647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3016-31b3-4853-a7fe-9ac299f2c555 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.292187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bb3016-31b3-4853-a7fe-9ac299f2c555 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.293451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 52/104] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:22.298658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.298948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.299229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.319273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:22.321318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:22.332173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6dd7711-a07c-4cbe-9232-1b1d54b483b7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:22.368539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:22.369411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.369841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.370281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] PATH_INFO: `/v3/users/5cf63fb9b3b945788239500950cea5d0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.386146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] RBAC: Authorizing `identity:get_user(user_id=5cf63fb9b3b945788239500950cea5d0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:22.387828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:22.390195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 53/105] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:22 2022] GET /identity/v3/users/5cf63fb9b3b945788239500950cea5d0 => generated 289 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:22.391219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.407818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:22.421302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:22.461635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 53/106] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:22.462608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.511827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af93716e-d3b0-44e1-9d6c-0d667b52bcdf None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:22.572719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:22.573541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.573829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.574103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.576783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:22.578239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:22.580117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 54/107] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:22.580671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.591851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3923b6c-2e08-4530-8323-a8acba513fb4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:22.656902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:22.657539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.657826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.658100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.736796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:22.754518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:22.767835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9549f0-533f-436d-9c5d-4c220650877a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:22.782786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:22.784296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c585cd71-4433-424d-8d9f-1fb219b3218f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:22.828797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:22.829453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.829739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.830013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.850530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:22.856050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:22.856930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:22.857700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:22.859654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 55/108] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:22.860574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.919064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 54/109] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:22.921097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dff744-54dc-4638-87e8-9f859c08f623 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:22.921500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dff744-54dc-4638-87e8-9f859c08f623 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:22.922139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dff744-54dc-4638-87e8-9f859c08f623 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:22.923585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 55/110] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:22.924393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dff744-54dc-4638-87e8-9f859c08f623 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:22.989057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ed4f74-c1f7-42ff-9841-98e6718b256f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.030912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.031569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.031852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.032003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.032134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.032858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.034395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.049019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dff744-54dc-4638-87e8-9f859c08f623 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.050038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.066647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 56/111] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:08:22 2022] POST /identity/v3/projects => generated 302 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:23.068044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.086441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.087231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.087654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.088134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.089059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:23.089496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.091132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6959e70-3b5c-4208-83f5-6f53b82d5b98 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.101953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 56/112] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.103578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7929c5-99dd-4904-99ca-93fdab93df20 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.103910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7929c5-99dd-4904-99ca-93fdab93df20 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.104295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7929c5-99dd-4904-99ca-93fdab93df20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.126124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ff7929c5-99dd-4904-99ca-93fdab93df20 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:23.127664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ff7929c5-99dd-4904-99ca-93fdab93df20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:23.215901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 57/113] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:23.217470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d18aa-e769-42fb-9b86-2ed085e81e19 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.217868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d18aa-e769-42fb-9b86-2ed085e81e19 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.218294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54d18aa-e769-42fb-9b86-2ed085e81e19 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.219883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 58/114] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:23 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:23.220739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d18aa-e769-42fb-9b86-2ed085e81e19 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.227100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75b78ae-d49d-4163-a6f9-73a9c6a56af3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.277673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.279057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.279057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.279057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.280536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.281983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8200790f-3853-43c4-9d8a-ff3e6e1debb8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.284283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 57/115] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.284950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.285260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.286412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.306649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:23.309472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:23.340121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.368318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54d18aa-e769-42fb-9b86-2ed085e81e19 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.381924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 58/116] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:23.383609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.387777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.416837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.417715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.418343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.419002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.421472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.423730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.425633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 59/117] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.426631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.515262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d562a88-657e-4bc8-a0e9-2942bcd3c2d3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.551968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.552626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.552911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.553187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.553287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ebe4b86-a479-4740-a3b0-9f2f6c0789ff None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.566524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:23.573531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:23.574852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:23.575635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 59/118] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:23.576241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.594843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.595814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.596098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.596376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.597193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=5cf63fb9b3b945788239500950cea5d0) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:23.597505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.598994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.605631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 60/119] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/role_assignments?user.id=5cf63fb9b3b945788239500950cea5d0&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=39f201b228e249ffa03e81043cf7f946 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.606253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.621362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.646585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.702461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029821fb-52d0-4609-b2ae-ecb95cc6e73e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdad53d-a2e5-41dc-b70f-6f7cff2d6c3d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.741477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.742703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.742990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.743291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.755314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:23.758607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.759340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.759722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.760074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:23.760901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:23.761726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:23.762890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:23.763508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.763574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 60/120] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:23.763574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.763700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1619c715-0fbe-4032-8bcc-a6146babb606 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.765399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 61/121] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.771838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dd08-8859-422b-a1f0-1be81b647e0f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.772126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dd08-8859-422b-a1f0-1be81b647e0f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.772406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a7dd08-8859-422b-a1f0-1be81b647e0f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.773744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 62/122] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:23.775632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7dd08-8859-422b-a1f0-1be81b647e0f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.879026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ebd8-b637-4e22-af8b-ed00658e6858 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.884455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.894815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:23.923665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a7dd08-8859-422b-a1f0-1be81b647e0f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:23.929937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.930619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.930973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.931259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.932037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:23.932349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.933896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.945185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 61/123] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.946569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:23.968088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:23.968865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.969176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.969459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:23.970249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:23.970559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:23.972221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d5b5ab-64e3-428c-81de-9378883f6516 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:23.982627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 63/124] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:23 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:23.988867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:23.989166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:23.989456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.014392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:24.015948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:24.058131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff690cfe-0f47-4c41-be76-e0f3eea4c311 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.093225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.094264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.094769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.095244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/da5b1c90e892464380eacee9a29f0449/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.097687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, user_id=da5b1c90e892464380eacee9a29f0449, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.098053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 64/125] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:24.099653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.104272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.116320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 62/126] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:23 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/da5b1c90e892464380eacee9a29f0449/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:24.120506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.191461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:24.201384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:24.271042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a33d6a-a4ea-4a8a-8c56-8b158792db5f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.274230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58013156-c675-45fa-bdee-26ae4dd06ded None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.310674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.311750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.312242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.312753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.314477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.316290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.318691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 63/127] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:24.319901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.366214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.366907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.367194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.367502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.368953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.371561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.373306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 65/128] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:24.374087 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.447002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30d03-bd4c-401e-88c5-060a7769d383 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.450624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:24.500570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1a29f7-1664-464d-b585-fa7d741834f7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.521837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.523154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.524201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.524491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.547058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:24.554874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:24.555798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:24.556587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 64/129] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:24.559600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.567225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.567963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.568248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.568527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.569825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.572086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.579198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:24.580033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.599498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:24.707178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b1a8ec5-4ebe-4780-904e-7462734a1538 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.745185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.745948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.746240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.746518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.747736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.748523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15fa335f-bd20-421b-92cd-3a213d877733 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.749268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.751359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:24.752044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.787668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:24.800142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.800785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.801067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.801513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.802074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:24.802383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.803798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.813218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 67/132] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:24.813839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:24.865259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757baf30-86a8-438e-a8c1-19075578c3fb None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:24.903575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:24.904305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:24.904595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:24.904874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/6692a4b4a7a24f49983927b5b63565f7/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:24.906876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, user_id=6692a4b4a7a24f49983927b5b63565f7, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:24.908469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:24.927402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 66/133] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:24 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/6692a4b4a7a24f49983927b5b63565f7/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:24.928640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:25.008307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:25.063319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2f631c-aa61-4565-afe2-ec7be7b9f911 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:25.079875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:25.105162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:25.105905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.106399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.106694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.108108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:25.109586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:25.111539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 67/134] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:24 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:25.120810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:25.133547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4405b-810e-4fc8-8c44-571257966538 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:25.265533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ee02ee-3a6e-4910-a258-e940dbf6007d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:25.303729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:25.304474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.305260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.305365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:25.305566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] PATH_INFO: `/v3/projects/02d404154c9f45158242839af9898f0e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.306094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.306329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.306609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/6692a4b4a7a24f49983927b5b63565f7/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.306704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] RBAC: Authorizing `identity:get_project(project_id=02d404154c9f45158242839af9898f0e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:25.308308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:25.310014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 68/135] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:25 2022] GET /identity/v3/projects/02d404154c9f45158242839af9898f0e => generated 312 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:25.315684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:25.316247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, user_id=6692a4b4a7a24f49983927b5b63565f7, role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:25.317721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8891a-42da-4f04-bb6c-7355430ab495 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:25.344917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 68/136] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:24 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/6692a4b4a7a24f49983927b5b63565f7/roles/cce34632ed81427db845cc21f77efac0 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:25.351830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:25.437840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ccc603-07e2-4abc-b195-714356cb38ed None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:25.476140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:25.476932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.477223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.477504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] PATH_INFO: `/v3/projects/02d404154c9f45158242839af9898f0e/users/6692a4b4a7a24f49983927b5b63565f7/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.479637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] RBAC: Authorizing `identity:create_grant(project_id=02d404154c9f45158242839af9898f0e, user_id=6692a4b4a7a24f49983927b5b63565f7, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:25.481109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b80c02-5e2d-40d8-9e16-ee22e646274a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:25.499442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 69/137] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:25 2022] PUT /identity/v3/projects/02d404154c9f45158242839af9898f0e/users/6692a4b4a7a24f49983927b5b63565f7/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:08:25.644684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250853-2b43-45ee-97f5-f6fe8d8ee2f1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.645248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250853-2b43-45ee-97f5-f6fe8d8ee2f1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.645248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54250853-2b43-45ee-97f5-f6fe8d8ee2f1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.646626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 69/138] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:25.659832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98024-d7e9-43d1-9bb4-5208220ea873 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.660175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98024-d7e9-43d1-9bb4-5208220ea873 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.660469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f98024-d7e9-43d1-9bb4-5208220ea873 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.682923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b7f98024-d7e9-43d1-9bb4-5208220ea873 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:25.684291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b7f98024-d7e9-43d1-9bb4-5208220ea873 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:25.760066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 70/139] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:25.788307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54250853-2b43-45ee-97f5-f6fe8d8ee2f1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:25.933173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54250853-2b43-45ee-97f5-f6fe8d8ee2f1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:25.965987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:25.967306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba10fd-cfd7-43ca-847d-b3d2807a1d3c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.967871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba10fd-cfd7-43ca-847d-b3d2807a1d3c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.969545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba10fd-cfd7-43ca-847d-b3d2807a1d3c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.969972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 71/140] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:25.974029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:25.974683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.975368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.975927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.978051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:25.978260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:25.978923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:25.979469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:25.979897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:25.981730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 70/141] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:25.987607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:26.021628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:26.023521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:26.053580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:26.096355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490f6f96-8e98-48d2-9469-59f6ba583688 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:26.135448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 72/142] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:26.155338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:26.156032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:26.156319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:26.156601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:26.157410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:26.158883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:26.168505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:26.206095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 71/143] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:25 2022] POST /identity/v3/users => generated 283 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:26.312568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bcf234-74b5-4044-ac95-679c4fa10780 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:26.354562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:26.355231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:26.355543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:26.355879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:26.373248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:26.378288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:26.379144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:26.379953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 73/144] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:26 2022] GET /identity/v3/roles/reader => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:26.392222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:26.493925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:26.550652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1053ffdf-de28-4916-b06c-bd763b247f66 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:26.598974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:26.599641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:26.599928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:26.600208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:26.600948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:26.601299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:26.602687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:26.615533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 72/145] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:26.618049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:26.637463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:26.793061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e79eb4-7126-4ad1-82c1-3010fafe407a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:26.839216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:26.839964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:26.840252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:26.840532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] PATH_INFO: `/v3/users/5cf63fb9b3b945788239500950cea5d0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:26.841745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] RBAC: Authorizing `identity:get_user(user_id=5cf63fb9b3b945788239500950cea5d0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:26.843202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:26.845007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 74/146] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:26 2022] GET /identity/v3/users/5cf63fb9b3b945788239500950cea5d0 => generated 289 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:26.850415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:26.970033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972533c3-fe8f-47fa-9e86-43a7934668c3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:26.993789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:27.022517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:27.023228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.023528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.023808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.028104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:27.029598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d692b222-3f2e-4df0-8afe-04b69a8ac435 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:27.031300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 73/147] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:26 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:27.040485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:27.117806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc5c7-d146-49ff-915b-136cf3a51e74 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.118116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc5c7-d146-49ff-915b-136cf3a51e74 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.118285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594cc5c7-d146-49ff-915b-136cf3a51e74 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.120743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 74/148] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:27 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:27.127376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.127376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.127376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.135268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:27.147816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:27.148093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:27.162937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc81d8-e997-46e1-a4f5-2490d0678eeb None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:27.210849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:27.212365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.212365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.212365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/5cf63fb9b3b945788239500950cea5d0/roles/39f201b228e249ffa03e81043cf7f946` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.214849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, user_id=5cf63fb9b3b945788239500950cea5d0, role_id=39f201b228e249ffa03e81043cf7f946)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:27.216337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:27.236406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 75/149] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:27 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/users/5cf63fb9b3b945788239500950cea5d0/roles/39f201b228e249ffa03e81043cf7f946 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:27.279643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 75/150] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:27.358192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:27.525821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7744ed45-9c5b-4797-8b92-e5d0180eb38a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:27.584241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:27.584925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.585216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.586037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.588539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:27.600601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:27.606216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:27.607255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:27.608049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:27.611854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 76/151] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:27.615442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:27.742812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37b545a-519c-4a50-ad3c-4c05cf08451c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:27.756725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:27.787065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:27.787659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.787946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.788227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.789550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:27.789917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:27.791293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:27.803460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 76/152] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:27.808958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:27.928737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08aa085-18fd-4229-9377-3f9169dc73e8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:27.966949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:27.967642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:27.967926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:27.968204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] PATH_INFO: `/v3/users/da5b1c90e892464380eacee9a29f0449` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:27.969419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] RBAC: Authorizing `identity:get_user(user_id=da5b1c90e892464380eacee9a29f0449)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:27.970879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:27.972900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 77/153] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:27 2022] GET /identity/v3/users/da5b1c90e892464380eacee9a29f0449 => generated 247 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:27.978240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:28.096319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9599eaa8-82d5-431c-a2bc-9972ce48d429 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:28.167126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:28.167805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:28.168088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:28.168367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:28.231253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:28.262844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:28.264349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:28.387222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 77/154] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:28.392832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:28.487397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:28.536939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ab7ab9-054f-4654-9881-fd5ad380e520 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:28.575499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:28.576156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:28.576442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:28.576717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:28.577913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:28.579372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28238ad0-0330-4c0c-bdcf-de5dbe33a05c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:28.581383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 78/155] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:28 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:28.586988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:28.721385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:28.754760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579d1e57-c5f8-452b-b5ef-a98a43c7734c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:28.802558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:28.803183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:28.803482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:28.803764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:28.805230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=da5b1c90e892464380eacee9a29f0449) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:28.805544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:28.806997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b324cdce-5bfc-4289-a46c-7a109212da8a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:28.815883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 78/156] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:28 2022] GET /identity/v3/role_assignments?user.id=da5b1c90e892464380eacee9a29f0449&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:28.901818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb3a8-0680-4251-8193-610aa1ca041a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:28.902375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb3a8-0680-4251-8193-610aa1ca041a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:28.902375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43edb3a8-0680-4251-8193-610aa1ca041a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:28.903809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 79/157] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:28.913161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa25e5-a493-4d0b-add1-3723c5080160 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:28.914017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa25e5-a493-4d0b-add1-3723c5080160 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:28.914297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaa25e5-a493-4d0b-add1-3723c5080160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:28.941232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-aaaa25e5-a493-4d0b-add1-3723c5080160 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:28.942471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aaaa25e5-a493-4d0b-add1-3723c5080160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:29.028545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 79/158] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:29.056855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1741d7a-063c-4785-bddb-c2dbb4e98875 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.057196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1741d7a-063c-4785-bddb-c2dbb4e98875 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.057424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1741d7a-063c-4785-bddb-c2dbb4e98875 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.058993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 80/159] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:29.064581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.064873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.065151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.082567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1741d7a-063c-4785-bddb-c2dbb4e98875 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.091808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:29.093000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:29.231971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:29.240890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1741d7a-063c-4785-bddb-c2dbb4e98875 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:29.245834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 80/160] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:29.274712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:29.280765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.294527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:29.295189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.295529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.295773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.316405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-edd44fdb-e484-4af6-92a8-47c9a7c26d4b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:29.322793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:29.323745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:29.324514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:29.328161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 81/161] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:29.329440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ddc1d-c2b8-4be4-89e1-29152ba9c436 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.329728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ddc1d-c2b8-4be4-89e1-29152ba9c436 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.330006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9ddc1d-c2b8-4be4-89e1-29152ba9c436 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.331297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 82/162] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:29.335786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ddc1d-c2b8-4be4-89e1-29152ba9c436 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.427780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886e4c80-1f30-4bb3-9300-963278fc98d2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:29.502368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9ddc1d-c2b8-4be4-89e1-29152ba9c436 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:29.547856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:29.548554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.548837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.549112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.551827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:29.552458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.552742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.553017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.553754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:29.554673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:29.556029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58b6b2e5-21f9-4efa-a469-cb0ac0598f75 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:29.564920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 83/163] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:29.566278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.566570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.566850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.576858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:29.581972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:29.582811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:29.583928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 81/164] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:29.592015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.608735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:29.609999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:29.683790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:29.701731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:29.714436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 84/165] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:29.715293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.818279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cc52fa-2440-4202-85fd-55f1d54e0110 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:29.882379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:29.883037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:29.883332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:29.883649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:29.886403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:29.887897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:29.889533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 82/166] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:29.890308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:29.963657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ec87fe-ca0f-426a-ab34-390e1767e851 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.052169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.052814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.053095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.053374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.054152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:30.054461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:30.055982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:30.070512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 85/167] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 355 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:30.071589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:30.102450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a466399f-f28d-41de-9b00-dd7e4d7144dc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.164921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.171790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.194079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.195260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.195572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.196114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.224808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:30.238305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:30.239292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:30.240110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 83/168] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 350 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:30.240769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:30.269138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda54ae3-eb2b-4386-90d7-0cd9151aa794 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.329631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.330190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.330473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.330749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.439641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:30.441126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87bd434b-34c8-49d9-a313-c2623b17ca12 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:30.449729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b38e04-766c-4181-939a-84e9e91daba7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.520161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.532740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.533404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.533687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.533965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.535165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:30.536662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:30.538675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 84/169] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:30 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:30.539543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:30.618800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 86/170] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 548 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:30.620598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e713-f603-4295-9d93-455332bc5f66 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.621115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e713-f603-4295-9d93-455332bc5f66 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.621582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f8e713-f603-4295-9d93-455332bc5f66 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.623045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 87/171] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:30.624256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8e713-f603-4295-9d93-455332bc5f66 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:30.636810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.736566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-345a02cb-4c39-43d6-8679-f21140629da8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.772761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.773734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.774206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.774689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.775661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:30.776355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:30.778377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:30.787983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 85/172] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:30.789290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:30.801092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f8e713-f603-4295-9d93-455332bc5f66 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.885067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.885707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.886003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.886271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.897764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f71f04-4e53-4b6f-8614-37b98a5f8dc0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:30.915536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.915727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:30.936556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:30.937575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.938072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.938578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] PATH_INFO: `/v3/projects/02d404154c9f45158242839af9898f0e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.939968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] RBAC: Authorizing `identity:get_project(project_id=02d404154c9f45158242839af9898f0e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:30.941731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fdb07b-d234-4a6c-84c8-908579d3831f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:30.943715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 86/173] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:30 2022] GET /identity/v3/projects/02d404154c9f45158242839af9898f0e => generated 312 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:30.945645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:30.946411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:30.946873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:30.965263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:30.966973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:31.040634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 87/174] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:31.041514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.062929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.064562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.168983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 88/175] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 545 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:31.174285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.174827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aab981d-f8fd-4daf-9017-6b866d0087aa None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.191029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.213978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.214624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.214901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.215175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] PATH_INFO: `/v3/users/6692a4b4a7a24f49983927b5b63565f7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.216379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] RBAC: Authorizing `identity:get_user(user_id=6692a4b4a7a24f49983927b5b63565f7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.217821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0c9be-85fc-4857-b8a5-328e34617456 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 88/176] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/users/6692a4b4a7a24f49983927b5b63565f7 => generated 275 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f20c3-da0a-4130-a7fc-0aed160068f3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f20c3-da0a-4130-a7fc-0aed160068f3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408f20c3-da0a-4130-a7fc-0aed160068f3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 89/177] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:31.223472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f20c3-da0a-4130-a7fc-0aed160068f3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.229558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.283528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943e101-37e8-4d07-a15b-d1da2791803d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.317034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.317636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.317912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.318178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.318969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=02d404154c9f45158242839af9898f0e, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:31.319273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.320753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cb5b4e-c619-479c-8823-6f3b5e91e857 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.326572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 89/178] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=02d404154c9f45158242839af9898f0e&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:31.335923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588056-6d55-4807-9064-41139794afd9 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.335923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588056-6d55-4807-9064-41139794afd9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.335923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89588056-6d55-4807-9064-41139794afd9 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.335923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 90/179] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:31.335923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588056-6d55-4807-9064-41139794afd9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.342145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408f20c3-da0a-4130-a7fc-0aed160068f3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.384613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.385626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.386096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.386645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.402776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1357cdb3-f2a9-4ae9-b2e6-eda4c5673d33 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:31.408194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:31.409132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:31.409936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 90/180] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:31.415810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1dea-ea2f-4b5c-96a2-946189f2cfc8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.416095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1dea-ea2f-4b5c-96a2-946189f2cfc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.416394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fed1dea-ea2f-4b5c-96a2-946189f2cfc8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.417912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 91/181] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:31.418466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1dea-ea2f-4b5c-96a2-946189f2cfc8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.456714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89588056-6d55-4807-9064-41139794afd9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.467961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.469536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.500993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.501884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.502168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.502445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.504323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.505192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de532f8-342b-4b84-826a-d179a6fb2234 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.506873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 91/182] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:31.513005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0565af-addd-4461-91df-f75170e81c0c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.513381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0565af-addd-4461-91df-f75170e81c0c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.513666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0565af-addd-4461-91df-f75170e81c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.532297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9c0565af-addd-4461-91df-f75170e81c0c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:31.533499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9c0565af-addd-4461-91df-f75170e81c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:31.539226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fed1dea-ea2f-4b5c-96a2-946189f2cfc8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.575775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.576398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.576678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.576950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.602911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 92/183] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:31.604194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.604491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.604757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.628596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:31.629822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:31.645127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.646580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d2e7a4-5e4b-43ef-932e-d9d187716078 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.687351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.702338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 93/184] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:31.703184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.738877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.763630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 92/185] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:31.764877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.765165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.765437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.782697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:31.783896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:31.802632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d61d185-cff7-49a2-a546-af7a8b261ee5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.836429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.837049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.837330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.837604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.838331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:31.838632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.840142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.847756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 93/186] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:31.859553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 94/187] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:31.859650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.859752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.918117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.947661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e34a9f7-fa3e-4070-9bd4-1267311aa0f2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.955785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:31.959604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a005a08b-fee8-464e-b13b-4cc675bb7b7a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:31.980894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.981517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.981807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.982081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:31.982910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:31.983226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:31.984731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:31.990524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 94/188] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:31.991405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:31.994586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:31.995250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:31.995565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:31.995843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.007622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:32.012827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:32.013776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 95/189] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:32.020260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.118792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fad63da6-8ddf-43f4-869a-b45678aa5f46 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.150279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39573d6-95d0-4e4b-9707-94f8fdc3db93 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.159010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.159941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.160428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.160917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.166113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.175558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:32.181524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:32.182468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:32.183261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:32.183922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 95/190] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:32.187466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.197905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.198496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.198779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.199055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.200274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.201727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.203400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 96/191] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.205363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.236663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.294745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a610b1d6-1753-4d8a-8e80-52e0d92be7ca None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.308750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7064c980-c1a6-46c3-a8ed-a9fdbac846fa None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.327139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.327810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.328092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.328358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.341277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.341437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.341911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.342189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.342460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.342885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.343615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.344522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 96/192] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.345283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.349488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.351690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 97/193] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.352440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.410548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.461258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.463338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e51cbe8-9511-481d-9f41-d7af91044d29 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.468592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a95317-f1a3-477c-8add-8d477e0d47ab None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.497950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.498565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.498842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.499113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.499856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:32.500154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.500318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.500925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.501201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.501472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.501561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.502184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:32.502486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.503887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.509601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 97/194] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.510377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.512276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 98/195] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.513056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.609693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aad7e9-f14c-4b55-956f-d408889b9c94 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.610519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c51dbf-c549-44a8-8eae-a4c99b804da7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.625505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.643111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.643485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.643754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.644036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.644127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.644310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.644396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.644612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.645100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=6692a4b4a7a24f49983927b5b63565f7) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:32.645410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.646854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.653136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 98/196] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/role_assignments?user.id=6692a4b4a7a24f49983927b5b63565f7&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=cce34632ed81427db845cc21f77efac0 => generated 608 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:32.653891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.672975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.726299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.727857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.777805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2580d446-c8ac-45e3-bef0-cf6172ea3ba6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:32.819226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:32.820997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:32.821287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:32.821555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:32.822253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:32.823773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:32.847218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 99/197] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:32 2022] POST /identity/v3/users => generated 297 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:32.852559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.883011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 99/198] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:32.884841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:32.900070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.967319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:32.985314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec859ef-6685-4667-b5e8-02209c6a5f5d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.022092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34f273b-6cc7-40e7-bdb0-c785db094562 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.028279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.028943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.029217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.029490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.031142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.033345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.039690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 100/199] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:33.040544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.065219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.065872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.066157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.066430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] PATH_INFO: `/v3/users/5cf63fb9b3b945788239500950cea5d0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.068034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] RBAC: Authorizing `identity:get_user(user_id=5cf63fb9b3b945788239500950cea5d0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.071293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.073198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 100/200] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:32 2022] GET /identity/v3/users/5cf63fb9b3b945788239500950cea5d0 => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:33.073774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.172296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9913368f-c3ba-45ef-bc4a-51a50d8a17df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.176663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.213685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc47af7d-57bc-4752-9254-bbea26efcacc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.217980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.218636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.218912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.219183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.223247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.223633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.226778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 101/201] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:33.228012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.240817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.253450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.254368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.254805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.255183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.330578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.332239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.365746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fef628-b2de-4e24-8c96-a00010bb7531 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.416152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.417049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.417325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.417597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.442503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 101/202] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:33.443532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.471450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.475274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.509047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.511022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.559064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7735e4e-77f5-4f60-989a-3c81db1c1b87 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.595143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.596008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.596425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.596800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.597837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=1152380970de4ef8be4e446f791f17f8) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:33.598252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.599826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.606466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 102/203] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/role_assignments?user.id=1152380970de4ef8be4e446f791f17f8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:33.607462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.652699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 102/204] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:33.658740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.781792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.811852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:33.834755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede63a1c-1d80-432c-a31b-27a8dc70cb53 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.841778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97601834-9cf0-4d0c-80db-40aa07037a77 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:33.902446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.903097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.903374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.903825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.908158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.909755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.911586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 103/205] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:33.918232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:33.922421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:33.923067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:33.923340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:33.923641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:33.925507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:33.926942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:33.933708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 103/206] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:33.939708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:34.083624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a030e7c-ef5f-4e4b-9b71-cc8ffc6d67f9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:34.125959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35d5813-51ab-418d-8ee2-40666d0aa195 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:34.130545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:34.131231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:34.131556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:34.131834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:34.133182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=1152380970de4ef8be4e446f791f17f8) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:34.133489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:34.134942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c03453-1711-4283-af04-e94c4845cfa0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:34.152483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 104/207] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/role_assignments?user.id=1152380970de4ef8be4e446f791f17f8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=cce34632ed81427db845cc21f77efac0 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:34.192871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:34.193571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:34.193878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:34.194155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:34.195553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, scope.project.id=5217f721e7dc4ba1b007c1d77e414191, user.id=5cf63fb9b3b945788239500950cea5d0) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:34.196189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:34.197656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5abe2c7-a78b-4c55-b9dd-6b39f4aaa09a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:34.209072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 104/208] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:33 2022] GET /identity/v3/role_assignments?user.id=5cf63fb9b3b945788239500950cea5d0&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=39f201b228e249ffa03e81043cf7f946 => generated 608 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:35.028046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602822-1070-4f78-887b-f0e206717535 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.028546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602822-1070-4f78-887b-f0e206717535 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.028619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34602822-1070-4f78-887b-f0e206717535 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.029936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 105/209] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:35.035187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ebb89-faa7-4300-9357-6e09aa182fe5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.035512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ebb89-faa7-4300-9357-6e09aa182fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.035793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6ebb89-faa7-4300-9357-6e09aa182fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.043723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.056084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2e6ebb89-faa7-4300-9357-6e09aa182fe5 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:35.057268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2e6ebb89-faa7-4300-9357-6e09aa182fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:35.130950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 105/210] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:35.153870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34602822-1070-4f78-887b-f0e206717535 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.177515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.256582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09cad-3c9f-4ac6-9d75-64460d8c5dcd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.257118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09cad-3c9f-4ac6-9d75-64460d8c5dcd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.257190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d09cad-3c9f-4ac6-9d75-64460d8c5dcd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.258714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 106/211] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:35.259615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34602822-1070-4f78-887b-f0e206717535 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.263541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.263829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.264103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.282767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:35.283999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:35.295616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.296327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.296610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.296888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.307601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:35.312773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:35.313807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:35.314700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 106/212] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:35.319746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.366582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 107/213] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:35.386715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.405591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.426012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61de3f13-047d-4eda-b382-8a6f087e2914 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.463513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.464362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.464717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.465180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.466034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:35.466414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:35.468045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.469413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:35.481083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 107/214] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:35.486695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.515318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8ad7c7-7127-4096-959f-71f2fe67e7d3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.553948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.554845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.555129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.555475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.556645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:35.558845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:35.561225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 108/215] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:35.568764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.593993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad6f89-251d-47c1-a4b0-6ed5a4efa6da None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.627652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.628289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.628568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.628838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] PATH_INFO: `/v3/users/c7a05274f7554b74b1278f0c3ca33a65` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.641269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.644499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a05274f7554b74b1278f0c3ca33a65)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:35.645962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:35.648627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 108/216] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/users/c7a05274f7554b74b1278f0c3ca33a65 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:35.656338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:35.701815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919da605-b8a2-4bd3-b44b-df206dbe4bc7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.740699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.747328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.748084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.748888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.749681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.750751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:35.752901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e655178-e026-4207-b7e9-013318bfacc4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:35.773461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ebea03-0b67-4f41-a6a9-7663c1734be1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:35.787698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 109/217] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:35 2022] POST /identity/v3/users => generated 297 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:35.792601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971e768-d0ec-4f9b-9dfa-d75f0f51b87d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.792887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971e768-d0ec-4f9b-9dfa-d75f0f51b87d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.793167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5971e768-d0ec-4f9b-9dfa-d75f0f51b87d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.794555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 110/218] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:35.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.800450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.800699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.821862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:35.822180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:35.822980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:35.823357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:35.823945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:35.824465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:35.901367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 111/219] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:35.911478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:35.911478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:35.912300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:35.926138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.027235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 109/220] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:36.033127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.040128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6060e00f-9cb5-4d77-beec-4b4e5bd494fe None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.064574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:36.094973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.096869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.096869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.096869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.107965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:36.114583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:36.115741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:36.117544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 112/221] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:36.128542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.177210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:36.182146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd8061-0689-47dc-9222-9f0944cd6346 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.230762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.231608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.231892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.232167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.233715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:36.235559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:36.239116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 110/222] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:36.245352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.254969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a6951a-d546-467d-942c-4fd8acbb8b0c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.293775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.295167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.295673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.296181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.296981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:36.297358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:36.298829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:36.312600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 113/223] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:36.317997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.372946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c614b4d-ff7f-4554-a375-c53906e1ebf5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.374124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:36.419613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.420536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.420986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.421454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.422421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=c7a05274f7554b74b1278f0c3ca33a65) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:36.422894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:36.424937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383068a8-df20-4dc2-8038-6bcc2dcf63df None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:36.431719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 111/224] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/role_assignments?user.id=c7a05274f7554b74b1278f0c3ca33a65&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:36.479109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f4d56a-04c9-44f0-885b-68ad971dd9b3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.490896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5842-4bf9-42fd-985c-440454907978 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.492042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5842-4bf9-42fd-985c-440454907978 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.492491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f5842-4bf9-42fd-985c-440454907978 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.494163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 112/225] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:36.500017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.500616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.501062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.534229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:36.536004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:36.558964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.560024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.560502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.561103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.562555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:36.564324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:36.566571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 114/226] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:36.572125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.601513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:36.669333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 113/227] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:36.694655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.739189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43596ba-e8df-4d00-90d8-09de647a767e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.753151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:36.790086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.790744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.791028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.791305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.793354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:36.794813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:36.796539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 115/228] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:36.804475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.883544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe38ef7a-a2d5-47fb-8bda-e66b8aea8fce None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:36.939994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:36.940878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:36.941158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:36.941429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:36.953508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:36.958891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:36.959765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:36.961166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 114/229] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:36.970041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:36.988440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51426c1-daf1-4033-944c-24c5ff4dddc4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:37.011774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:37.053975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:37.055821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:37.056104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:37.056382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/1152380970de4ef8be4e446f791f17f8/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:37.064225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, user_id=1152380970de4ef8be4e446f791f17f8, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:37.065670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:37.101800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 116/230] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:36 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/1152380970de4ef8be4e446f791f17f8/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:37.120749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e0d4d-c974-48a9-aa0b-5ed1b9db2aa7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:37.189488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:37.190425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:37.190849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:37.191264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:37.192287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:37.192729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:37.194853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:37.210491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 115/231] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:37.216499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:37.285015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:37.375661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4924bd03-0e79-40a4-8e19-41a886147a32 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:37.414083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:37.414737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:37.415012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:37.415284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:37.417909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:37.419369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:37.425548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 117/232] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:37 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:37.426923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:37.551970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:37.571395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3bae4-3b07-4d18-a578-374931ff9d0a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:37.606082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:37.606730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:37.607013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:37.607290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:37.608485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:37.609933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e3ab4d-90df-4e22-ab29-737315b7c211 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:37.611949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 116/233] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:37 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:37.617580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:37.771678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bab68ff-49fa-4a21-8e22-55278b22c795 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:37.781806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:37.828772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:37.829379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:37.829670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:37.829952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/1152380970de4ef8be4e446f791f17f8/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:37.832296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, user_id=1152380970de4ef8be4e446f791f17f8, role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:37.833739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb33bd4-a5ff-4acf-bf25-33975a345382 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:37.865092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 118/234] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:37 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/1152380970de4ef8be4e446f791f17f8/roles/cce34632ed81427db845cc21f77efac0 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:38.410736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb522-ee97-463f-84c8-fe5fef5ab3d3 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.411241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb522-ee97-463f-84c8-fe5fef5ab3d3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.411241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2defb522-ee97-463f-84c8-fe5fef5ab3d3 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.412869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 117/235] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:38.418747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0611-1f99-4c32-a692-c4ecd41670f6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.419144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0611-1f99-4c32-a692-c4ecd41670f6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.419324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8d0611-1f99-4c32-a692-c4ecd41670f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.444894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eb8d0611-1f99-4c32-a692-c4ecd41670f6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:38.446355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb8d0611-1f99-4c32-a692-c4ecd41670f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:38.531453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 119/236] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:38.555309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defb522-ee97-463f-84c8-fe5fef5ab3d3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:38.636533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf91302-545d-477c-b752-517947eea47a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.636850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf91302-545d-477c-b752-517947eea47a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.637099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf91302-545d-477c-b752-517947eea47a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.638620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 120/237] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:38.648748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce8de-20bb-4801-8f39-9c33ea06d56b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.649297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce8de-20bb-4801-8f39-9c33ea06d56b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.649792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-974ce8de-20bb-4801-8f39-9c33ea06d56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.677637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-974ce8de-20bb-4801-8f39-9c33ea06d56b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:38.679851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-974ce8de-20bb-4801-8f39-9c33ea06d56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:38.736127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2defb522-ee97-463f-84c8-fe5fef5ab3d3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:38.763681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 121/238] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:38.781491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8f7-26dc-4de8-a214-c94a067303a3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.782464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8f7-26dc-4de8-a214-c94a067303a3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.782941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae88b8f7-26dc-4de8-a214-c94a067303a3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.784625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 122/239] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:38.789954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b8f7-26dc-4de8-a214-c94a067303a3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:38.802159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:38.803120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.803843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.804196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.805426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:38.806975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57a129e-6713-4df0-a752-54d59df7c6cf None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:38.808695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 118/240] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:38.809889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.814573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.814949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.833955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:38.836265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:38.839833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:38.852536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:38.915877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae88b8f7-26dc-4de8-a214-c94a067303a3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:38.917177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 119/241] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:38.919278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:38.963603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:38.968292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:38.969210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:38.969593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:38.983047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:38.990341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:38.991814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 123/242] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:38.997339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.051691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19f89a8-4bae-4668-860b-ac7da7958f67 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.094112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.095335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.095945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.096521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.097376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.099563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.101374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.121987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9e9e5d-6ad3-4eb9-9a12-91c4a07a6ee7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.134352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 120/243] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:38 2022] POST /identity/v3/users => generated 293 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:39.135532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.146374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.196938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.197608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.197887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.198159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.223585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:39.238373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:39.239339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:39.240151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:39.240752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 124/244] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:38 2022] GET /identity/v3/roles/member => generated 84 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:39.248684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.338701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c5e02-8e4b-4fb0-8608-4f5bc60b1672 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.392995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.393647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.393931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.394475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.395776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:39.396101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.397953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.410227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 121/245] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:39.416560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.421092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc82ac4a-ba43-4741-a7a4-7baf20e0f0ac None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.466530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.467581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.468083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.468556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.471296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:39.471982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.473796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.491678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 125/246] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:39.493895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.503200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.505225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.553131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82d0525-bc58-4542-abe3-912bc52ee8f9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.601520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.602564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.603045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.603544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] PATH_INFO: `/v3/users/0dcf0985a848493791be634aecc2b6d8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.620038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] RBAC: Authorizing `identity:get_user(user_id=0dcf0985a848493791be634aecc2b6d8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.622016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.624540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 122/247] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/users/0dcf0985a848493791be634aecc2b6d8 => generated 297 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:39.629635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.634182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8187adf-cde4-4921-9956-b5db8afdb231 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.678813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.682336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.682336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.682336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] PATH_INFO: `/v3/users/c7a05274f7554b74b1278f0c3ca33a65` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.682336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] RBAC: Authorizing `identity:get_user(user_id=c7a05274f7554b74b1278f0c3ca33a65)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.682640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5660fdc-a8f8-4b16-af31-461a05edcb93 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.684667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 126/248] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/users/c7a05274f7554b74b1278f0c3ca33a65 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:39.685959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4778-2c9b-4d7a-8621-6fc13f741b9b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.686238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4778-2c9b-4d7a-8621-6fc13f741b9b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.686511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208f4778-2c9b-4d7a-8621-6fc13f741b9b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.691787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 127/249] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:39.692603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f4778-2c9b-4d7a-8621-6fc13f741b9b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.758296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e06cc94-6f6d-4635-aa96-0e77dc18332b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.779378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.795783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:39.811487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.812116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.812393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.812668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.820021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208f4778-2c9b-4d7a-8621-6fc13f741b9b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:39.858819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:39.859491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.859862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.860136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.861304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.862839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c307a237-cfcc-4d46-a6f2-2517853394be None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.864806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 128/250] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:39.866053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:39.866335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:39.866710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:39.885281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:39.886567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:39.889736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:39.891274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:39.960973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 129/251] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:39.968268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:39.996037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 123/252] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:08:40.024762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.054871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.067906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.105033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ef230-077b-440a-87fa-6b9926bad1cf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.150025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.150712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.150989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.151264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/c7a05274f7554b74b1278f0c3ca33a65/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.153270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, user_id=c7a05274f7554b74b1278f0c3ca33a65, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:40.154732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:40.177018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 130/253] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:39 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/c7a05274f7554b74b1278f0c3ca33a65/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:40.177324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a243c67-3cf9-497c-af34-da69bca3149b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.177922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.244115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.244793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.245079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.245397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.248198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:40.249748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:40.251506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 124/254] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:40.260232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.399674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daecc416-3494-4a00-9cf0-407299616b6a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.425009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.433386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b45f86-5c90-40d3-a255-f1aafab1a8dd None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.467392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.480125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.481041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.481495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.481914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.499402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.501204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.504259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.504832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.505535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:40.505978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=0dcf0985a848493791be634aecc2b6d8) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:40.506496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:40.508627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806cc6f6-6b33-4328-8773-0a682fdd3328 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-358eea18-7e4d-4482-9ec8-e953e377cbbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:40.512711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:40.513617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:40.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 131/255] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:40.515076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3417d4-10ff-4f58-9b06-2025f86fc949 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.515076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3417d4-10ff-4f58-9b06-2025f86fc949 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.515203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3417d4-10ff-4f58-9b06-2025f86fc949 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.517089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 132/256] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:40.519021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 125/257] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/role_assignments?user.id=0dcf0985a848493791be634aecc2b6d8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=39f201b228e249ffa03e81043cf7f946 => generated 259 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:40.522042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3417d4-10ff-4f58-9b06-2025f86fc949 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.522789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.523274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.523753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.547274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:40.549063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:40.647850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 126/258] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:40.683336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.704238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3417d4-10ff-4f58-9b06-2025f86fc949 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.764344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.764999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.765277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.765548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.766283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:40.766585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:40.768142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:40.781903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 133/259] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:40.787601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.855450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.864712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:40.881660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf5e640-bcf4-42df-a239-aa3680a5a359 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.919446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da6a2ed-1cee-4d6c-a814-799066d6e84a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:40.948815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.949400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.949681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.949953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.969274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:40.980312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:40.981232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:40.982100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 127/260] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:40.984601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:40.984601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:40.984815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:40.984999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:40.986211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:40.987770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:40.988581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:40.989383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 134/261] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:40.995527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:41.155390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f39d47-5626-43cf-9d4b-cdcb10e05686 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:41.206915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:41.207485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:41.207768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:41.208042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:41.208770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:41.209090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:41.210976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:41.211945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff8d11d-5d5d-4ec5-8b81-624e05bf2a71 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:41.229478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 128/262] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:41.234705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:41.251566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:41.256901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:41.291281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:41.291976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:41.292264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:41.292543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:41.388009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92be6676-b55f-4c0c-8976-5457000dd796 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:41.451835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:41.452481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:41.452763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:41.453038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] PATH_INFO: `/v3/users/1152380970de4ef8be4e446f791f17f8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:41.454214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] RBAC: Authorizing `identity:get_user(user_id=1152380970de4ef8be4e446f791f17f8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:41.455750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:41.457721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 129/263] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:41 2022] GET /identity/v3/users/1152380970de4ef8be4e446f791f17f8 => generated 283 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:41.463744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:41.480237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:41.481779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:41.591876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:41.618509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8bce1e-3647-48f4-8ae2-d64f750c07a4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:41.677063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3af9-2300-47de-9300-146acee25523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:41.677704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3af9-2300-47de-9300-146acee25523 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:41.677982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3af9-2300-47de-9300-146acee25523 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:41.678284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177d3af9-2300-47de-9300-146acee25523 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:41.707974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 135/264] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 713 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:08:41.714731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:41.752318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:41.810209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d3af9-2300-47de-9300-146acee25523 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:41.812065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177d3af9-2300-47de-9300-146acee25523 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:41.910589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edacaba-07d1-4605-9d1c-9eb526feb8ea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:41.964955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:41.979190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:41.980439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:41.980911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:41.981414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:41.983721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:41.985375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:41.987444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 136/265] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:41 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:42.003215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 130/266] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 540 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:08:42.005063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.010413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3af9-2300-47de-9300-146acee25523 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.163218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:42.182721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e439e8f-fc0b-4547-bf93-2cab684523e0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.203592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177d3af9-2300-47de-9300-146acee25523 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.243075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.243496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.244180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.244458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.244728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.245526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=1152380970de4ef8be4e446f791f17f8) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:42.245832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:42.247265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:42.249315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.249596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.249867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.251142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:42.252582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077c776f-9737-4454-8f6b-f5a2c216f666 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:42.255670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 131/267] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:42.257503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f0bb-9485-40cb-a345-f1c740aab96a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.257785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f0bb-9485-40cb-a345-f1c740aab96a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.258054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c3f0bb-9485-40cb-a345-f1c740aab96a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.261935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 137/269] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/role_assignments?user.id=1152380970de4ef8be4e446f791f17f8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:42.267592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 132/269] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:42.268964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3953a-19f1-49c6-aaf1-39f51139e571 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.269250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3953a-19f1-49c6-aaf1-39f51139e571 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.269524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d3953a-19f1-49c6-aaf1-39f51139e571 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.273056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.299811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-96d3953a-19f1-49c6-aaf1-39f51139e571 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:42.302268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-96d3953a-19f1-49c6-aaf1-39f51139e571 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:42.397067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:42.450731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 133/270] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:42.452260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25f24-1fa9-4bc3-bbc1-627eca0f9620 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.452780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25f24-1fa9-4bc3-bbc1-627eca0f9620 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.452780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce25f24-1fa9-4bc3-bbc1-627eca0f9620 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.454198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 134/271] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:42.457808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cf20e-4917-4f2a-aa07-ac6ccdc2ca1e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.464620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.464906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.465181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.501474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:42.502582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:42.503600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.504240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.504632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.504918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.505726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=1152380970de4ef8be4e446f791f17f8) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:42.506036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:42.507703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:42.514422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 138/272] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/role_assignments?user.id=1152380970de4ef8be4e446f791f17f8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=cce34632ed81427db845cc21f77efac0 => generated 608 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:42.515033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.558673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:42.677057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 135/273] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:42.678028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4726f0c4-cc7f-4b16-bbfd-a44fdbb95cc5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.720868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.721900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.722411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.722966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.724139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.725139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:42.727632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:42.730808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 139/274] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:42.736699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.849462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:42.858554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a169d8-c2f6-4d92-b29d-af411a65a4f6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.866822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:42.868870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335b88f1-f1c3-4a46-80d5-4dda29634b06 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:42.895775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.896587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.897484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.897945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.906804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:42.907657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:42.908076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:42.908486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:42.909306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:42.909786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:42.911048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2f854d-5372-4821-9993-4b0bbdbb7af6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:42.915680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:42.916770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:42.917771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:42.920836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 136/275] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:42.927274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:42.937792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 140/276] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:08:42 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:08:43.025778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e22fd-4503-4e54-ad9c-2f8a4c3ce355 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.026105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e22fd-4503-4e54-ad9c-2f8a4c3ce355 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.026488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34e22fd-4503-4e54-ad9c-2f8a4c3ce355 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.028311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 141/277] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:43.033961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.034244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.034517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.041655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b401e8ce-e675-43de-8f11-8e295a04b899 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.060337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:43.061553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:43.078969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.079632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.080013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.080350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.081084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:43.081431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:43.082851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:43.093698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 137/278] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:43.100734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.132796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:43.169204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 142/279] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:43.196393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.244492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249edf6d-9910-414a-a109-2bf733d1fd0e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.285618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:43.287648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.288366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.288650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.288928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] PATH_INFO: `/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.311454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] RBAC: Authorizing `identity:get_user(user_id=fdf90a3977c84d05bb07ac7f3aa8b0cd)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:43.312977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:43.314719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 138/280] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd => generated 293 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:43.321099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.347038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49140363-4c2e-4c71-aa0c-d5b72c88b9f2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.399161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.399669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.399951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.400225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.411924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:43.417452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:43.418365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:43.419165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 143/281] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:43.424872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.433605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:43.454609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595bf575-48d5-40fa-b70f-5f004af40e0a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.524141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.524919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.525453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.525673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.526579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, user.id=fdf90a3977c84d05bb07ac7f3aa8b0cd, scope.system=all) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:43.526995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:43.528567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:43.541352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 139/282] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/role_assignments?user.id=fdf90a3977c84d05bb07ac7f3aa8b0cd&scope.system=all&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 226 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:43.546483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b4f38-965a-42bd-9a68-32595c58ee69 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.582005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:43.595360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.596376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.596837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.597289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.598223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:43.598720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:43.600431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17523dad-818e-4e18-b7e7-e19f9889d987 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:43.610174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 144/283] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:43.616529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.755930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e37a15-6a00-4b2c-b66b-ef5f7ca6eaab None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.756176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e37a15-6a00-4b2c-b66b-ef5f7ca6eaab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.756937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e37a15-6a00-4b2c-b66b-ef5f7ca6eaab None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.758287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 145/284] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:43.765788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.765928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.766106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.768943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a94ae4-8c34-44b0-aabf-0c4d696efc07 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:43.785904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:43.787085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:43.819588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:43.819588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:43.819588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:43.819588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] PATH_INFO: `/v3/users/c7a05274f7554b74b1278f0c3ca33a65` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:43.820047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] RBAC: Authorizing `identity:get_user(user_id=c7a05274f7554b74b1278f0c3ca33a65)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:43.820047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:43.820047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 140/285] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/users/c7a05274f7554b74b1278f0c3ca33a65 => generated 297 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:43.826597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.869655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 146/286] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:43.872754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:43.891853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:43.991605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3254d2e7-bc46-4485-baf4-4e1c684bfb44 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.014496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66678b7-1cea-45c2-a6cb-d2e12926b2ca None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.039839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:44.059525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.060895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.061171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.061440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.079187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:44.080920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.081536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.081815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.082088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:44.087285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:44.088194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:44.088995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 147/287] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:44.094219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.211767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.213245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.234571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:44.309385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4840a30b-3caa-4f2f-bf35-e66d082dec75 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.335181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 141/288] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 509 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:44.340611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.404123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.404760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.405042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.405324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.406054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:44.406365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.407784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.422101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 148/289] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:44.429705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.459986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1579c97a-999a-45b4-ad48-efccd35f66df None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.480784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:44.502332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.503273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.503754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.504203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.505575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.508684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.510838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 142/290] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:44 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:44.517485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.539450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:44.590714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754743b1-5f38-49cb-8b87-d103fb1f363c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.634210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.634862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.635143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.635444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] PATH_INFO: `/v3/users/0dcf0985a848493791be634aecc2b6d8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.639128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] RBAC: Authorizing `identity:get_user(user_id=0dcf0985a848493791be634aecc2b6d8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.640665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.642324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 149/291] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:44 2022] GET /identity/v3/users/0dcf0985a848493791be634aecc2b6d8 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:44.649026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.674386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fb6833-247f-490f-afa5-7b830e4f24b3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.742120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.742750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.743031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.743304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.744144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=c7a05274f7554b74b1278f0c3ca33a65) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:44.744456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.745897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.763341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 143/292] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:44 2022] GET /identity/v3/role_assignments?user.id=c7a05274f7554b74b1278f0c3ca33a65&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:44.809941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d71f07-7633-49c6-a688-51c0023674bd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:44.825909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:44.854525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:44.855124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:44.855407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:44.855715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:44.856938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:44.858455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6157c0b7-13c3-4480-abbc-dfb3a067e563 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:44.860461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 150/293] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:44 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:44.868544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:44.985819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:45.050041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5b3d45-804c-438e-b46c-302eb50a450c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:45.086544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:45.087226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.087532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.087808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/0dcf0985a848493791be634aecc2b6d8/roles/39f201b228e249ffa03e81043cf7f946` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.089771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, user_id=0dcf0985a848493791be634aecc2b6d8, role_id=39f201b228e249ffa03e81043cf7f946)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:45.091200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f86e29a6-1946-4c09-a9dc-468fd3001933 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:45.108281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 144/294] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:08:44 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/users/0dcf0985a848493791be634aecc2b6d8/roles/39f201b228e249ffa03e81043cf7f946 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:45.436618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a87452-776b-4be2-af51-b5ef3a5b6095 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.436989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a87452-776b-4be2-af51-b5ef3a5b6095 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.437184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a87452-776b-4be2-af51-b5ef3a5b6095 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.438673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 151/295] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:45.444641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4d09e-70a3-431e-a398-ba5a6ab84171 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.445150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4d09e-70a3-431e-a398-ba5a6ab84171 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.445889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d4d09e-70a3-431e-a398-ba5a6ab84171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.464912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d0d4d09e-70a3-431e-a398-ba5a6ab84171 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:45.466882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d0d4d09e-70a3-431e-a398-ba5a6ab84171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:45.545352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 145/296] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:45.568555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a87452-776b-4be2-af51-b5ef3a5b6095 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:45.679451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a87452-776b-4be2-af51-b5ef3a5b6095 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:45.712023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae276-7c05-486b-9581-cf319cf81cc9 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.712646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae276-7c05-486b-9581-cf319cf81cc9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.713166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719ae276-7c05-486b-9581-cf319cf81cc9 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.714891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:45.715473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 146/297] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:45.715659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.719609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.719609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.721060 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.721549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.722025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.729866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:45.735584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:45.736678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:45.737636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:45.745316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 152/298] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:45.748501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:45.749225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:45.750588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:45.800725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:45.802232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:45.859379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 147/299] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:45.878071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f3bcfd-4e4b-42ca-8729-1de8ba193772 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:45.885894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:45.920580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:45.922026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:45.922618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:45.923173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:45.924565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:45.925360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:45.926947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:45.937993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 153/300] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:45.944913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.008397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a627698a-a468-4dfa-b39a-2871363d8c70 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.041119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.041753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.042028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.042301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.043456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.044886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efea8252-d703-49cd-8d17-2c5058ed05c2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.050823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 148/301] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:46.050823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4eb01-e006-4eaa-9b93-9050fed92be5 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.050823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4eb01-e006-4eaa-9b93-9050fed92be5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.050823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac4eb01-e006-4eaa-9b93-9050fed92be5 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.050823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 149/302] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:46 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:46.052313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4eb01-e006-4eaa-9b93-9050fed92be5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.068326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c89bb74-5c4f-4cd7-b89e-807f269ec344 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.071709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:46.098033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:46.106233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.106875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.107159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.107450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] PATH_INFO: `/v3/users/b43ba30731cd4ed5a8355f7211bc86bb` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.121272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] RBAC: Authorizing `identity:get_user(user_id=b43ba30731cd4ed5a8355f7211bc86bb)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.122802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad6ba9-7b6a-41f8-b42b-37faa4371b38 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.124840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 154/303] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:45 2022] GET /identity/v3/users/b43ba30731cd4ed5a8355f7211bc86bb => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:46.126949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.126949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.126949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.147970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:46.149166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:46.168998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac4eb01-e006-4eaa-9b93-9050fed92be5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.204700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.205682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.206213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.206699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] PATH_INFO: `/v3/groups` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.207522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.209003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.221521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 155/304] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:46.231573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.238698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 150/305] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Jul 31 14:08:46 2022] POST /identity/v3/groups => generated 229 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:08:46.245287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.352113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:46.353430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:46.357158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f3966-1d91-4f1e-8e7b-281fc9929e57 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.363909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fe6d08-e854-4f63-9386-dec27da3df46 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.395002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.395703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.395981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.396268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.408291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:46.410536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.411460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.411962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.412548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:46.413543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:46.414402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:46.415186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:46.415849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, user.id=b43ba30731cd4ed5a8355f7211bc86bb, scope.system=all) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:46.415925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 151/306] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:46 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:46.419663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.422869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.422985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.426871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 156/307] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 31 14:08:46 2022] GET /identity/v3/role_assignments?user.id=b43ba30731cd4ed5a8355f7211bc86bb&scope.system=all&role.id=39f201b228e249ffa03e81043cf7f946 => generated 226 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:46.535449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fc1265-a83f-44bb-afbf-9960a5a6f51f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.576155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.576789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.577072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.577352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.578089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:46.578401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.579810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.589529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 152/308] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:46 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:46.596508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.703894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4ba023-d1de-4204-af01-1e6ceb8ba073 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.739500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.740152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.741069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.741351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] PATH_INFO: `/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.742541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] RBAC: Authorizing `identity:get_user(user_id=fdf90a3977c84d05bb07ac7f3aa8b0cd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.744016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cb9a25-251a-4545-8008-df22f421430a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.745943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 157/309] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:46 2022] GET /identity/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd => generated 293 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:46.751336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:46.773594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:46.855051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b3df0-e31b-4982-b138-f3eb5ecf1762 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:46.891897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:46.892938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:46.893421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:46.893895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] PATH_INFO: `/v3/system/users/fdf90a3977c84d05bb07ac7f3aa8b0cd/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:46.903444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fdf90a3977c84d05bb07ac7f3aa8b0cd, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:46.903444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79cbf6d-1b46-4d0f-96fc-de696f7e6eff None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:46.919643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 153/310] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Jul 31 14:08:46 2022] PUT /identity/v3/system/users/fdf90a3977c84d05bb07ac7f3aa8b0cd/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:47.259257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af23-f8d5-48a7-85b7-8a4f49e110b6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:47.259642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af23-f8d5-48a7-85b7-8a4f49e110b6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:47.259843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fd4af23-f8d5-48a7-85b7-8a4f49e110b6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:47.261192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 158/311] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:47.266784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909758f1-82b6-46cc-8659-822807150fad None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:47.267082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909758f1-82b6-46cc-8659-822807150fad None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:47.267364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909758f1-82b6-46cc-8659-822807150fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:47.286837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-909758f1-82b6-46cc-8659-822807150fad None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:47.288642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-909758f1-82b6-46cc-8659-822807150fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:47.357015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 154/312] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:47.380697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4af23-f8d5-48a7-85b7-8a4f49e110b6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:47.407146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:47.487053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fd4af23-f8d5-48a7-85b7-8a4f49e110b6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:47.527066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:47.527805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:47.528091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:47.528367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:47.539896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:47.545347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:47.546211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:47.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 159/313] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:47.557030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909758f1-82b6-46cc-8659-822807150fad None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:47.576127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:47.671108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909758f1-82b6-46cc-8659-822807150fad None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:47.706622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:47.707595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:47.708047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:47.708487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:47.709489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:47.709965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:47.711539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:47.719920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 155/314] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:47.730290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:47.820837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:47.837052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445bd66b-2256-4ad1-9d54-98b79ed0dbeb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:47.874426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:47.875083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:47.875365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:47.875676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] PATH_INFO: `/v3/users/0dcf0985a848493791be634aecc2b6d8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:47.878299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] RBAC: Authorizing `identity:get_user(user_id=0dcf0985a848493791be634aecc2b6d8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:47.880408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:47.885658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 160/315] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:47 2022] GET /identity/v3/users/0dcf0985a848493791be634aecc2b6d8 => generated 297 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:47.891497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:47.989470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.015995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe10d6b6-4821-4ed5-b719-b3b0409d456b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.050863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:48.051813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.052274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.052753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.132168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:48.133640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:48.220127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.256917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 156/316] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:48.262307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.364893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5692ae34-a092-4695-8918-f362a411c796 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.399466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:48.400116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.400395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.400665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.401821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:48.403262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a95b002-dd03-4fe7-9f05-a76737cfd285 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:48.405275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 161/317] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:48 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:48.411041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.506705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a2c71-7d49-4a29-9455-bd44713b62fc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.507079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a2c71-7d49-4a29-9455-bd44713b62fc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.507243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0a2c71-7d49-4a29-9455-bd44713b62fc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.508863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 162/318] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:48.513905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.514411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.514921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.516695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9808b5b-6288-4455-9943-04a91faa8f60 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.533710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:48.535097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:48.554840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:48.555511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.555831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.556108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.556901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b, user.id=0dcf0985a848493791be634aecc2b6d8) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:48.557207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:48.558085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.558944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cacaff3a-1966-4c3a-9785-933a3459e1a6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:48.571444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 157/319] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:08:48 2022] GET /identity/v3/role_assignments?user.id=0dcf0985a848493791be634aecc2b6d8&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=39f201b228e249ffa03e81043cf7f946 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:48.571444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f23e-2f3d-41dc-b1ee-2ceca0887829 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.571444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f23e-2f3d-41dc-b1ee-2ceca0887829 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.571444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8340f23e-2f3d-41dc-b1ee-2ceca0887829 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.571444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 158/320] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:48.574689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.574981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.575263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.595235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:48.596503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:48.637767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 163/321] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:48.647397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.663107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.686342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 159/322] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:48.710285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.784098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc693609-c93b-41a2-9cdb-2fe54852e93f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.822978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:48.823643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.823923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.824195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.832590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34d74b9-3e9c-4c96-8e1b-5ee1464afa8a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.836798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:48.841168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:48.842226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:48.843084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 164/323] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:48.848548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.862187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.879721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:48.880656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:48.880993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:48.881935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:48.883334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:48.884812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:48.886413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 160/324] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:08:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:48.891937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:48.935975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:48.963314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef9447-8328-448b-9120-c216d87fb401 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:48.999388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c5ed1e-ed08-41a3-93c1-33847c684e4e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.001176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.002295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.002295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.002295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.003019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:49.003318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.004727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.013920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 165/325] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:49.019502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.036672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.037749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.038029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.038300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] PATH_INFO: `/v3/groups` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.038962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.040411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3af9e182-a05c-45f1-a1d5-37a88810b61e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.056856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 161/326] 10.222.0.40 () {60 vars in 1134 bytes} [Sun Jul 31 14:08:48 2022] POST /identity/v3/groups => generated 226 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:08:49.057981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5344-ec53-49c2-9417-4a7fe435b853 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.058267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5344-ec53-49c2-9417-4a7fe435b853 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.058539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5a5344-ec53-49c2-9417-4a7fe435b853 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.060250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 162/327] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:49.065845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.066133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.066411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.085835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:49.087053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:49.096837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:49.135681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fca4c3-95a9-48eb-a3d3-f4c416aec572 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.158588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 163/328] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:49.174657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.175318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.175625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.175901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] PATH_INFO: `/v3/users/b43ba30731cd4ed5a8355f7211bc86bb` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.177133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] RBAC: Authorizing `identity:get_user(user_id=b43ba30731cd4ed5a8355f7211bc86bb)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.178717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.180581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 166/329] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:49 2022] GET /identity/v3/users/b43ba30731cd4ed5a8355f7211bc86bb => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:49.184509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.185920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.214637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:49.303251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f20f46-c2e0-4268-9e64-fa2df5e52ac7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.315526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f65f1e-e4dc-4d87-b533-cea14b51787a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.350561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.351212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.351571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.351846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.361050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.362233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.362517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.362794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] PATH_INFO: `/v3/system/users/b43ba30731cd4ed5a8355f7211bc86bb/roles/39f201b228e249ffa03e81043cf7f946` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.365815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:49.365941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b43ba30731cd4ed5a8355f7211bc86bb, role_id=39f201b228e249ffa03e81043cf7f946)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.366490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:49.370476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:49.371409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:49.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:49.375577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 164/330] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:49.378749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.382769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 167/331] 10.222.0.40 () {58 vars in 1334 bytes} [Sun Jul 31 14:08:49 2022] PUT /identity/v3/system/users/b43ba30731cd4ed5a8355f7211bc86bb/roles/39f201b228e249ffa03e81043cf7f946 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:49.487918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:49.504360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5913638-7d11-4da1-a1e0-3a60b8d2a5d3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.543115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.544105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.544600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.544970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.545855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:49.546304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.547792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.557654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 165/332] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:49.563968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.570735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:49.681352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a1ddf-1c8c-4049-916e-23d301305ffc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.719109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.719799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.720075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.720349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] PATH_INFO: `/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.721530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] RBAC: Authorizing `identity:get_user(user_id=fdf90a3977c84d05bb07ac7f3aa8b0cd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.722983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a267489-2796-4742-ba5c-8a6c52bcc8c9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.725455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 168/333] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:49 2022] GET /identity/v3/users/fdf90a3977c84d05bb07ac7f3aa8b0cd => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:49.731226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:49.852402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00507e74-9a7a-44e0-bfda-f4a50ff5564f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:49.898101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:49.899345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:49.899721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:49.900070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:49.900877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] RBAC: Adding query filter params (role.id=dcc526de6b3241fa9ae5f10eb055621a, user.id=fdf90a3977c84d05bb07ac7f3aa8b0cd, scope.system=all) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:49.901184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:49.902835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399ac75a-d79e-4905-b825-203c425d1173 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:49.929806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 166/334] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 31 14:08:49 2022] GET /identity/v3/role_assignments?user.id=fdf90a3977c84d05bb07ac7f3aa8b0cd&scope.system=all&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 510 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:51.525981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c304-2669-428d-8bc2-04544d2efc6b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:51.526462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c304-2669-428d-8bc2-04544d2efc6b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:51.526462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc06c304-2669-428d-8bc2-04544d2efc6b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:51.528032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 169/335] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:51.533311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:51.534192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:51.534662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:51.560483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:51.562027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:51.564430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:51.640759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 167/336] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:51.680728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06c304-2669-428d-8bc2-04544d2efc6b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:51.759963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:51.799281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc06c304-2669-428d-8bc2-04544d2efc6b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:51.835711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:51.836358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:51.836634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:51.836963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:51.850046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a53cffe8-e602-4e90-9273-4edd140748b8 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:51.855453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:51.856315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 31 14:08:51.857094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:51.859144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 170/337] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:51.864084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:51.911083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f66f4-0dc7-4dee-a9b0-43b4491833ef None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:51.911403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f66f4-0dc7-4dee-a9b0-43b4491833ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:51.911649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101f66f4-0dc7-4dee-a9b0-43b4491833ef None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:51.913226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 171/338] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:51.920861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad870d3-5017-43e9-84c7-bf8dfa8811dc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:51.921440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad870d3-5017-43e9-84c7-bf8dfa8811dc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:51.921902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad870d3-5017-43e9-84c7-bf8dfa8811dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:51.945890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0ad870d3-5017-43e9-84c7-bf8dfa8811dc None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:51.947675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0ad870d3-5017-43e9-84c7-bf8dfa8811dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:51.997717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5be4f3-d442-438d-a870-83152e4fe980 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.047669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.048378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.048575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.048831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.049612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:52.049860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:52.051223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d41153e7-a41a-425b-8e8d-86c93c9f68c9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:52.056903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.065407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 168/339] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:52.067284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0d92-4adc-4ea9-84cc-72687da833de None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.067641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0d92-4adc-4ea9-84cc-72687da833de None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.068128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5a0d92-4adc-4ea9-84cc-72687da833de None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.071917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 169/340] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:52.074216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0d92-4adc-4ea9-84cc-72687da833de None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.094053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 172/341] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:52.095337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.095666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.095946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.136527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:52.137736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:52.184265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5a0d92-4adc-4ea9-84cc-72687da833de None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.219013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.219688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.219966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.220237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] PATH_INFO: `/v3/users/b43ba30731cd4ed5a8355f7211bc86bb` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.221398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] RBAC: Authorizing `identity:get_user(user_id=b43ba30731cd4ed5a8355f7211bc86bb)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:52.222891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:52.225093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 170/342] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/users/b43ba30731cd4ed5a8355f7211bc86bb => generated 293 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:52.225856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.229066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.249668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 173/343] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:52.250547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.319919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.355756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ba4430-8cc8-43ea-bb6e-1d997c960c19 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.397513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.398210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.398495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.398768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.414115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd9887a-75a1-4dc1-a0d1-20216b53d4d0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.414979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:52.422186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:52.423125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:52.423935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:52.426704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 171/344] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:52.428412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.459981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.460593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.460874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.461148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.461952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] RBAC: Adding query filter params (role.id=39f201b228e249ffa03e81043cf7f946, user.id=b43ba30731cd4ed5a8355f7211bc86bb, scope.system=all) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:52.462264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:52.463745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86c597-74b3-4ccc-aba9-07ec52a57af2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:52.476715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 174/345] 10.222.0.40 () {56 vars in 1308 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/role_assignments?user.id=b43ba30731cd4ed5a8355f7211bc86bb&scope.system=all&role.id=39f201b228e249ffa03e81043cf7f946 => generated 510 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:52.478044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f08f57-b203-4f04-9db0-e5cb8b9ab4ab None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.478334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f08f57-b203-4f04-9db0-e5cb8b9ab4ab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.478614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f08f57-b203-4f04-9db0-e5cb8b9ab4ab None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.479905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 175/346] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:52.480614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f08f57-b203-4f04-9db0-e5cb8b9ab4ab None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.544897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41f718d-4ec1-41a3-9c3a-271b76dec941 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.568652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.578796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.591863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.592656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.592977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.593199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.619113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-687fb5c8-1551-45a4-91eb-4474680b1d06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:52.626798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:52.627802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:52.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 172/347] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:52.629384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.630258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.634823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.656809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:52.658003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:52.669093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f08f57-b203-4f04-9db0-e5cb8b9ab4ab None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.731721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.732450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.732825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.733103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.733856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:52.734180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:52.734718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 173/348] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:52.735655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:52.735845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.749322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 176/349] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:52.754702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.865114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b5b65-4abf-4ce8-8865-874d8eb590fa None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.884665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.895360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9e8665-9882-4050-99c9-d13beb540e1d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:52.907653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.908467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.908806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.909088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:52.909817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:52.910175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:52.911290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:52.913299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:52.922427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 174/350] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:52.925804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:52.941400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:52.942653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:52.944592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:52.946545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.040039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.042262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.046393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c74e578-2ab0-4865-a678-c4f5615a5bbf None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.083984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.085072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.085554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.086042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.097660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:53.103809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:53.104698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:53.105478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:53.107987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 175/351] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:53.109022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.152255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 177/352] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:53.153170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.166908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.168617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.253920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cc1094-d468-4fa2-a834-4ca6c30841eb None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.278993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde59b03-1afb-40d1-a9cd-7709a0c65141 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.295192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.295859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.296290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.296569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.318735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.319861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.320666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.320946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.321927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:53.322232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.324032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.336070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 178/353] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:53.337082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.397066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.398958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.452219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.459688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.489954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29a59cd-f997-4ad6-82bc-6d94e2047674 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.541659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.542312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.542594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.542915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.544018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 176/354] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:53.544547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.544865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.545994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.548163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 179/355] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:53.552308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.676783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b9326-6ba3-46a3-ae72-27689a3bc6ba None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.680311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55394bd0-77bb-473a-89fe-6342faf0f0c5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.714810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.715536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.715816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.716095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.716954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.719249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.719249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.719249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.722637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.722637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.724158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 180/356] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:53.725466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.728835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.742769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:53.795788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.797252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9cf3f8-1d47-4bf4-a4d7-7b40f70d6f92 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.857874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b469f71-bace-489c-9a22-2fe7cb6bc694 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:53.895866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:53.897500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.897991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.898269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.905724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:53.907941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8239a74a-a5ad-47f3-839c-ad5e95135a1e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:53.910377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 181/357] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:53.911962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f77b6-2eb8-4f30-bc60-ff66a4e646f1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.912388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f77b6-2eb8-4f30-bc60-ff66a4e646f1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.912916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15f77b6-2eb8-4f30-bc60-ff66a4e646f1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.914220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 182/358] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:53.914912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f77b6-2eb8-4f30-bc60-ff66a4e646f1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.917026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 177/359] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:53.918613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811af200-2b6b-44fb-bca9-f5e44eac9149 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:53.918901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811af200-2b6b-44fb-bca9-f5e44eac9149 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:53.919315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811af200-2b6b-44fb-bca9-f5e44eac9149 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:53.920646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 178/360] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:53.921498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811af200-2b6b-44fb-bca9-f5e44eac9149 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:53.970746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.008432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.036717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15f77b6-2eb8-4f30-bc60-ff66a4e646f1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.045173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811af200-2b6b-44fb-bca9-f5e44eac9149 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.080697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.081351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.081630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.081900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.083105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.085202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4863af47-b782-4dd7-b3b1-622655d82d69 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.086577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.091494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 183/361] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.091635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.091635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.091635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.091833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d8b4-db75-4a70-afa8-5faf1a046619 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.091903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=5217f721e7dc4ba1b007c1d77e414191) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:54.091969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d8b4-db75-4a70-afa8-5faf1a046619 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.091969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97d8b4-db75-4a70-afa8-5faf1a046619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.092084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.092084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.096782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 179/362] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:08:53 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=cce34632ed81427db845cc21f77efac0 => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.097457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.108670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9e97d8b4-db75-4a70-afa8-5faf1a046619 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:54.110257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9e97d8b4-db75-4a70-afa8-5faf1a046619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:54.198710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 184/363] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:54.200395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.200888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.201381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.221320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:54.222450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:54.237329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5974125a-4fd7-4899-b1b9-d32b7730bf04 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.241067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.285010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900c40c-8782-4311-9901-f8900462075e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.285678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c40c-8782-4311-9901-f8900462075e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.285940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c40c-8782-4311-9901-f8900462075e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.286209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a900c40c-8782-4311-9901-f8900462075e None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.288277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900c40c-8782-4311-9901-f8900462075e None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.289749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a900c40c-8782-4311-9901-f8900462075e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.291504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 180/364] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.292276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900c40c-8782-4311-9901-f8900462075e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.305684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 185/365] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:54.306870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.309617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.414918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a900c40c-8782-4311-9901-f8900462075e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.438616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5d171f-d879-4d60-8af9-559ef6796981 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.454519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.455251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.455562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.455945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.456750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=5217f721e7dc4ba1b007c1d77e414191) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:54.457072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.458498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.466010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 181/366] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.466841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.482788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271b252-6591-4057-83bd-665617395515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.484121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271b252-6591-4057-83bd-665617395515 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.484656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271b252-6591-4057-83bd-665617395515 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.485799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8271b252-6591-4057-83bd-665617395515 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.501546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8271b252-6591-4057-83bd-665617395515 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8271b252-6591-4057-83bd-665617395515 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:54.509746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:54.510739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:54.511560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:54.512192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.514196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 186/367] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:54.514387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271b252-6591-4057-83bd-665617395515 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.598454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab57a5-85ce-445d-acf8-ceb1fb3b5bf3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.598831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.645756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.647232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.647674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.648111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.649652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.651927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.654794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 182/368] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.655672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.662569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8271b252-6591-4057-83bd-665617395515 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.712733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.713463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.713668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.713944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.728358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:54.735275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:54.736201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:54.736986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 187/369] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:54.737753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.794983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e6b46-49c7-446f-9c29-4ad08b8e67b8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.797608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.834917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.835672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.835953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.836230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.836979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:54.837316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.838725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.848487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 183/370] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.849292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.860338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950a1804-9da3-4a22-a74b-abe2efdda66f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.886490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:54.900294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:54.901584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:54.902219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:54.902832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:54.904030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:54.904483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:54.906099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:54.912319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 188/371] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 260 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:54.913289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:54.961326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52f4a12-80df-4805-b35f-df8041fcfb55 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:54.999279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.000873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.001160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.001445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.002182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:55.002817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.004388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.018069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 184/372] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:55.019158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.029509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58ae76c-6ff6-4e0c-94c8-48716febad43 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.052626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.070688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.071316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.071625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.071898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.143894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.145547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf453cec-cd7e-4e15-90d3-5974747359a5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.172779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.174262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.196006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.197625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.198222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.200011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.311289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.313066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.338503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 189/373] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:55.348913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.373132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.451689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.470010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da84a373-8390-42b3-8a9d-0abe823538dc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.516560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 185/374] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:08:55.521695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.521695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.521695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.521695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.522623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.523399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.524311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.530527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 190/375] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:55.536454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.645289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af20be8-916b-42d4-afe5-a71df4100773 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.677254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.708116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.709265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.709759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.710247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] PATH_INFO: `/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.717699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] RBAC: Authorizing `identity:get_project(project_id=0e21ca24dfa94435a41c5ab2bd838cb9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.717699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.717699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 186/376] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:55.719662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffdf95f-6ef4-496c-b728-7105519f85dd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.725009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.775317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.776362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.776859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:55.777367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:55.778793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:55.782461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:55.784389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 191/377] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:55.790599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:55.820536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:55.919375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf628885-9a41-471e-b964-480f4e7556da None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.934753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9adaa9-c5d3-4991-9b2c-aa31b221d094 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:55.996681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.997671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:55.998620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.999450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:55.999923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.000400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] PATH_INFO: `/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.000875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.001973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.002964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:56.005036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:56.007242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62996776-8d8e-4139-a98f-14388695b827 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:56.014303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] RBAC: Authorizing `identity:get_group(group_id=6ea8d6adbb6540e39d4bfdc25ea25da1)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:56.017273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:56.022426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 187/378] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1 => generated 229 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:56.025676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 192/379] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:08:55 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=cce34632ed81427db845cc21f77efac0 => generated 260 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:56.026726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:56.100145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:56.147466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9698247-c244-4ddc-b67d-f6377c2fc407 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:56.184483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:56.185042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:56.185320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.185591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.186408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] RBAC: Adding query filter params (group.id=6ea8d6adbb6540e39d4bfdc25ea25da1, role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=0e21ca24dfa94435a41c5ab2bd838cb9) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:56.186716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:56.188208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df37d22-c64a-4e1c-81c9-9e8d26224aa6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:56.199549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 188/380] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:08:56 2022] GET /identity/v3/role_assignments?group.id=6ea8d6adbb6540e39d4bfdc25ea25da1&scope.project.id=0e21ca24dfa94435a41c5ab2bd838cb9&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 260 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:56.506290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a39fd-8556-44cd-9297-10b2154b5ace None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:56.506594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a39fd-8556-44cd-9297-10b2154b5ace None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.506795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a39fd-8556-44cd-9297-10b2154b5ace None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.508417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 193/381] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:56.513245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:56.513531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.513810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.540429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:56.541632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:56.621343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 189/382] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:56.647052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a39fd-8556-44cd-9297-10b2154b5ace None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:56.750518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:56.769396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a39fd-8556-44cd-9297-10b2154b5ace None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:56.810876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:56.811552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:56.811845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:56.812104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:56.825933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:56.831406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:56.833607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:56.835654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 194/383] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:56.843445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:56.907698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:56.964735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7bda6e-f6e2-4812-aa64-dbc1095a1d38 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:57.013838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:57.014464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.014738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.015007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.015759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:57.016060 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:57.017412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:57.030748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 190/384] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:57.036444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.156404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f149c818-97e9-4d75-8024-9d5a9bce2347 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:57.198877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:57.199589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.199873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.200147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.201329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:57.202760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:57.204398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 195/385] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:57 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:57.209984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.335257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f5dd32-cb9e-4427-a679-e5585f6ed869 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:57.357637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:57.384521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:57.385522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.386000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.386470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.392112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:57.393779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed98494-3efe-4790-b23e-e8b2c04ad2e5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:57.396015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 191/386] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:57 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:57.401476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.429589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:57.433111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee808-38f8-47b0-bdb9-23fa5b840fe7 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.433640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee808-38f8-47b0-bdb9-23fa5b840fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.434134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cee808-38f8-47b0-bdb9-23fa5b840fe7 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.436063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 192/387] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:57.441311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.441845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.442352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.462810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:57.464401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:57.587213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a38cda4-978f-4fbf-b188-013a014cf929 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:57.590189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 193/388] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:08:57.626996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.637664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:57.638368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.638647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.638920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/groups/ea263ee495164034ad36a888f006ca88/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.648379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, group_id=ea263ee495164034ad36a888f006ca88, role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:57.650191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea027d0-be89-4a5c-8544-1b92e3f509a8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:57.682658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 196/389] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 31 14:08:57 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/groups/ea263ee495164034ad36a888f006ca88/roles/cce34632ed81427db845cc21f77efac0 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:57.716015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006c2f-f535-4419-9781-0f4c9d3116ea None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.716411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006c2f-f535-4419-9781-0f4c9d3116ea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.716700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84006c2f-f535-4419-9781-0f4c9d3116ea None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.718379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 197/390] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:08:57.733170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.733520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.733718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.755607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff5ebe7-1a06-4d27-af77-b9799c7a450a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:57.768237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:57.769451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:57.771504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:57.816414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:57.817138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:57.817419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:57.817691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:57.842869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:57.852875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:57.853988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:57.854789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:57.856928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 194/391] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:57.864659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.902127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 198/392] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:57.926894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:57.930525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:58.031913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7071909c-eea8-4b5e-9b82-5f43ca7674e8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.093528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5062aaed-025d-413d-b60d-f1854c4c9ae2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.098552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.100096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.101397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.101888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.107257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:58.107835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.109438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.130064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 195/393] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:58.136517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.177219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.178480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.182068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.200324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:58.207592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:58.208566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:08:58.209359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:58.219286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:58.220233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 199/394] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:08:58.231904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.333101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:58.357148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae20ff2-df35-4617-aa76-7da41eb25d1a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.402489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-091d2284-c893-4dce-9b6b-1469f4c5b71d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.441042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.441704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.441987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.442263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.448122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.449563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.451180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 196/395] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:58 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:58.456638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.460747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.462011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.462508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.462976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.463942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:58.464448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.466055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.477110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 200/396] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:08:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:58.483617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.631179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc985a0a-1dc3-4dc2-952f-e1bf4e54b925 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.634057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:58.656435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00490ad2-d0b2-4b51-b611-8701fb25fbb8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.691180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.692202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.692705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.693241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.695981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.697631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.701959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 197/397] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:58 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:58.705170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.707778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:58.721211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.722390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.722874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.723368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.725501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.727268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df97768-fff4-4f30-89cf-c920ab681efd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.729240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 201/398] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:58 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:58.730616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a0c5b-822e-4da6-8700-06eea4165cf0 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.735340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a0c5b-822e-4da6-8700-06eea4165cf0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.736412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04a0c5b-822e-4da6-8700-06eea4165cf0 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.737976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 202/399] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:58.738927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a0c5b-822e-4da6-8700-06eea4165cf0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.828728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9bcb9b-298c-480c-be9d-ef7f9a18a82a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.870002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.870670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.870950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.871223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191/groups/ea263ee495164034ad36a888f006ca88/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.872664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04a0c5b-822e-4da6-8700-06eea4165cf0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:58.873214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] RBAC: Authorizing `identity:create_grant(project_id=5217f721e7dc4ba1b007c1d77e414191, group_id=ea263ee495164034ad36a888f006ca88, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.874634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a3b750-2b05-4e7e-a02a-72240b77cc2a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.887884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 198/400] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 31 14:08:58 2022] PUT /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191/groups/ea263ee495164034ad36a888f006ca88/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:58.889062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf35de-acd2-44f0-a88d-4e58b9e3801c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.889354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf35de-acd2-44f0-a88d-4e58b9e3801c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.889632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23bf35de-acd2-44f0-a88d-4e58b9e3801c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.916662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:58.917440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.917765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.918103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.919253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-23bf35de-acd2-44f0-a88d-4e58b9e3801c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:58.919368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:58.920850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-23bf35de-acd2-44f0-a88d-4e58b9e3801c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:58.920967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331b82af-9c27-4eb7-a678-f72919792a6d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:58.922804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 203/401] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:58 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:58.924435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682811-c43f-4864-8dad-dd9785b6ab49 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:58.924736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682811-c43f-4864-8dad-dd9785b6ab49 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:58.924999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4682811-c43f-4864-8dad-dd9785b6ab49 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:58.926411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 204/402] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:08:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:08:58.931679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682811-c43f-4864-8dad-dd9785b6ab49 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:58.955030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.003834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.011744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 199/403] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:59.013635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.013921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.014196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.046499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:08:59.051725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:08:59.054851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4682811-c43f-4864-8dad-dd9785b6ab49 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.104714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.106191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.106909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.107363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/groups/ea263ee495164034ad36a888f006ca88/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.110606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, group_id=ea263ee495164034ad36a888f006ca88, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:59.112993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:59.132837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 205/404] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 31 14:08:58 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/groups/ea263ee495164034ad36a888f006ca88/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:08:59.134001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.135130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 200/405] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:08:59.164544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.325727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.338674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bdd3f3-e9cd-49ee-9c99-b6aba73bd68a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.347059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa93ef4-8f1f-49d2-a0a0-8a6d43f59556 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.374325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.388145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.389259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.389758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.390299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.392429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.393360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.393710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.394375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.405665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:59.408228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:59.411531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:59.413047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:08:59.413850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:08:59.414436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:08:59.415227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:08:59.416103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 206/406] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:59.418974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 201/407] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:08:59.427667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.428624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.583254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d202ca21-e092-4427-9206-bd0085927760 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.605459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44974cf-8fc1-4b4e-b290-d90f90ccb883 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.633266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.634232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.634695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.635136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.636163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:59.636641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:59.638259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:59.647844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.648465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.648983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.649332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.650192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:08:59.650570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:59.652223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:59.655976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 207/408] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:59.661115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.666510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 202/409] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:08:59.667784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.691199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.760040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.827465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51081ebe-5229-433e-ba94-f3d2fd757348 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.859470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:08:59.860499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:08:59.860927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:08:59.861347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:08:59.862659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:08:59.864288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:08:59.871183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 203/410] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:08:59.872566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:08:59.929592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d005d9d-ea7e-4e4b-b793-50ed1676e8ba None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:08:59.961251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:08:59.981473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499c941-038f-4427-a45e-e953aefa6830 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.017981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.019081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.019574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.019680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.020140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.020449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.020799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.021665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] PATH_INFO: `/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.022756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.024387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-962e87ac-3784-44e0-82d1-81ab24240156 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.026452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 204/411] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:00.027917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851aaf7b-96eb-4f89-b991-40a8ff12ddec None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.028293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] RBAC: Authorizing `identity:get_group(group_id=6ea8d6adbb6540e39d4bfdc25ea25da1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.028612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851aaf7b-96eb-4f89-b991-40a8ff12ddec None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.029064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-851aaf7b-96eb-4f89-b991-40a8ff12ddec None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.029967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.030949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 205/412] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:00.032062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 208/413] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:08:59 2022] GET /identity/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1 => generated 229 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:00.036320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.036894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.037185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.037459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.058461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:00.060429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:00.159507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 206/414] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:00.160726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.190868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b73b65e-2dd9-475d-85fd-91c449918548 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.205796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:00.230381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.231396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.232042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.232565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/groups/ea263ee495164034ad36a888f006ca88/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.235104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] RBAC: Authorizing `identity:create_grant(project_id=d02c4dd209ae4d3abdbc8e841c368c4b, group_id=ea263ee495164034ad36a888f006ca88, role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.236782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.253245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 209/415] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 31 14:09:00 2022] PUT /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b/groups/ea263ee495164034ad36a888f006ca88/roles/cce34632ed81427db845cc21f77efac0 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:00.254433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.293558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:00.377475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b7ec7-9aba-43d1-9cde-3b9c2fe5a50f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.382340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04017a96-6a40-4254-9c16-36f78342ac7f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.426836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.427976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.428465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.428941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.441062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:00.448015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:00.449073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:00.450095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:00.452441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 210/416] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:00.453122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.453798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.454081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.454359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] PATH_INFO: `/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.460338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] RBAC: Authorizing `identity:get_project(project_id=0e21ca24dfa94435a41c5ab2bd838cb9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.461803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.463940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 207/417] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:00 2022] GET /identity/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9 => generated 344 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:00.466916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.470099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.501353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:00.584308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8586e46b-7d9f-4988-ac16-51f1961c5615 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.590419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029a4af9-2b15-4a58-841b-02669329d464 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.633544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.634888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.636150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.637100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.637854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.638750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:00.639645 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.640098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.641647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.642072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.642489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] PATH_INFO: `/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9/groups/6ea8d6adbb6540e39d4bfdc25ea25da1/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.646616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] RBAC: Authorizing `identity:create_grant(project_id=0e21ca24dfa94435a41c5ab2bd838cb9, group_id=6ea8d6adbb6540e39d4bfdc25ea25da1, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.648912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:00.658572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 211/418] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:00.665319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:00.684098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 208/419] 10.222.0.40 () {58 vars in 1442 bytes} [Sun Jul 31 14:09:00 2022] PUT /identity/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9/groups/6ea8d6adbb6540e39d4bfdc25ea25da1/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:09:00.778267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:00.787017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c88ce0-4ec2-4995-b839-cbbc57493aec None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:00.825836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:00.826905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:00.827387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:00.827899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:00.903779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:00.905707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:01.010848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 212/420] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:09:01.011469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.017587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.031313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.131150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f599759-b61b-4890-8192-af8486b35e37 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.168354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.169013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.169295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.169572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.170742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:01.172216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc5d00e-7159-4862-b1be-0470d249d192 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:01.174178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 209/421] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:01.179836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.285238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6ac2f-dac4-4670-87d4-9e4c5f89ccab None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.285538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6ac2f-dac4-4670-87d4-9e4c5f89ccab None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.285722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb6ac2f-dac4-4670-87d4-9e4c5f89ccab None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.286090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7424d68-41d7-4565-b638-f905a8938970 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.292110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 210/422] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:01.297357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.297645 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.297922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.325897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.326551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.326833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.327113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.329996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:01.331455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:01.333538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 213/423] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:01.334524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:01.335934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:01.339452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.411093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.463821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a92df6d-0159-495f-bd38-ee583f895c3d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.489987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 211/424] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:01.514966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.515635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.515918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.516193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.517061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=5217f721e7dc4ba1b007c1d77e414191) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:01.517374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:01.518867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b02f28-6089-47c9-9b01-2e59dd0bfd31 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:01.522112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.526303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 214/425] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=cce34632ed81427db845cc21f77efac0 => generated 611 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:01.539055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.613974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313255d3-55cf-4e94-b1e9-c2c8c05b0a74 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.614320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313255d3-55cf-4e94-b1e9-c2c8c05b0a74 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.615114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-313255d3-55cf-4e94-b1e9-c2c8c05b0a74 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.616662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 215/426] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:01.627777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.628104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.628342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.652243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec270b3-9232-48a2-8d7d-12759d8eb36a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.656995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:01.658246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:01.705921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.707304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.707621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.707902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.724516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:01.731825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:01.732990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:01.733981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 212/427] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:01.739290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.753161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 216/428] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:01.793627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.817547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.862556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43886af4-6dfd-49ea-b79b-cfd500ffcd1f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.876944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:01.907940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.908571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.909004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.909340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.910434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] RBAC: Adding query filter params (name=member) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:01.910777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:01.912999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:01.924148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 213/429] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:01.930194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:01.934793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dc4839-2557-4a85-92fe-b536ca98520b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:01.972762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:01.973770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:01.974801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:01.975308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:01.986797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:01.992346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:01.993218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 31 14:09:01.994018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:01.996623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 217/430] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:02.003457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.053757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23b291d-9994-4fcf-8963-e6c3c655b47d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.082191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.092320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.092939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.093216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.093487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.132611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b57cfe-add6-4a79-9104-59cc22853e09 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.157684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.180941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.182932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.186596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.187511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.188025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.188499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.189449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] RBAC: Adding query filter params (name=member) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:02.189977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.192153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.204540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 218/431] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:02.210315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.300069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 214/432] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:09:02.305966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.334578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2da381-6a16-4ebb-9534-ab8fc3134a81 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.336489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.380469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.381096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.381380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.382169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.407674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af461728-425f-4a2d-8276-53fbac4c1b36 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.446326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.459547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.460755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.461581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.462261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] PATH_INFO: `/v3/projects/5217f721e7dc4ba1b007c1d77e414191` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.463896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] RBAC: Authorizing `identity:get_project(project_id=5217f721e7dc4ba1b007c1d77e414191)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.466138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.468242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 215/433] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/projects/5217f721e7dc4ba1b007c1d77e414191 => generated 298 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:02.473773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.475168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.476866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.596272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.598030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 219/434] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:09:02.599615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.605095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a4c2db-185d-4813-91af-5208ca1b1307 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.651472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.652748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.653554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.653982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.655289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.657699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b499597e-8f9f-47f9-a6e6-7927d89404c9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.660703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 216/435] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:02.663264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e87a1-5598-47f6-b4f0-e230735fc351 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.663846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e87a1-5598-47f6-b4f0-e230735fc351 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.664387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e87a1-5598-47f6-b4f0-e230735fc351 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.666466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 217/436] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:02.667635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e87a1-5598-47f6-b4f0-e230735fc351 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.683949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.706929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d52045-87d1-4325-afa5-95e995ed31fd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.744037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.744695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.744972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.745246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.746829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.748495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb314490-f299-4ecc-a2c5-7d7bb76555d6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.750244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 220/437] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:02.751362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.751695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.751971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.769882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:02.771093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:02.787886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e87a1-5598-47f6-b4f0-e230735fc351 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.830231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:02.830911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:02.831129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:02.831404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:02.832230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=5217f721e7dc4ba1b007c1d77e414191) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:02.832540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:02.834597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:02.848148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 218/438] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=5217f721e7dc4ba1b007c1d77e414191&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 611 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:02.849196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.866337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 221/439] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:02.887941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:02.890198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:02.982198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f47e667-7010-4cb6-bbf7-592a743a5e39 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:02.996407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.023159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.023538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84ad726-b430-4fe1-ad93-47fa963426b9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.024368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.024652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.024942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.027592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.028732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.030558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 219/440] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:03.036220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.066438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.067141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.067441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.067831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.083029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:03.088898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:03.089785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:03.090565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 222/441] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:03.097184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.157908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.168747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9340b3-20b0-4638-a190-f40cd8d3f40a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.218315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.219015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.219219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.219531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.220344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=dcc526de6b3241fa9ae5f10eb055621a, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:03.220656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.222092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7cab5-ef55-480a-a8e9-c662d54ed4bc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.232478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 220/442] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=dcc526de6b3241fa9ae5f10eb055621a => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:03.232642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da360fe4-5b7c-4b24-a61a-08b543f7e71d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.234173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654cce-8965-458c-8ef9-62fcc7fb949c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.234462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654cce-8965-458c-8ef9-62fcc7fb949c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.234838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27654cce-8965-458c-8ef9-62fcc7fb949c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.236606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 221/443] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:03.243336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.243743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.244027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.266352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:03.267646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:03.275644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.276290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.277113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.277113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.277573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:03.277884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.279371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.282002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.294247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 223/444] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:03.299807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.365622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 222/445] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:03.386989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.434205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c97e33-176f-4643-b0b5-bb010b4c80c3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.476536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.477174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.477468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.477743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.499328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.514069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed86e56-2675-4852-8a98-1ad74dc5b48f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.559592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.563302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.565202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.567402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.568578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.568860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.569133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.581201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:03.586789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:03.588014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:03.588903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 223/446] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:03.593724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.678948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 224/447] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:09:03.685144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.711399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3154ffb-d35d-48a5-83f4-ed0511f34f6c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.744621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.745237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.745519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.745796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.746514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:03.746820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.748238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.756578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 224/448] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:03.762044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.775914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.781533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.785624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6922d017-7b0d-4fd3-9124-91275d2c3a40 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.821916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.822549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.822831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.823106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] PATH_INFO: `/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.824293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] RBAC: Authorizing `identity:get_project(project_id=d02c4dd209ae4d3abdbc8e841c368c4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.825722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.827373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 225/449] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/projects/d02c4dd209ae4d3abdbc8e841c368c4b => generated 302 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:03.832690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:03.861319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2c860e-6f5d-418a-9483-2f58b943b96a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.897942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.898563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.898839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.899113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.940090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e00552-f403-4df8-b5a0-aea63d5f3f25 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:03.964499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.965952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.972588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:03.973223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:03.973502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:03.974297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] PATH_INFO: `/v3/groups/ea263ee495164034ad36a888f006ca88` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:03.975488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] RBAC: Authorizing `identity:get_group(group_id=ea263ee495164034ad36a888f006ca88)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:03.976916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:03.978500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 226/450] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/groups/ea263ee495164034ad36a888f006ca88 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:03.980251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:03.984447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:04.018216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:04.063945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 225/451] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 31 14:09:04.069391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:04.083270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35163fbf-5f81-4ca8-b7bd-9a77a29a62c4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:04.116632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:04.117239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:04.117519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:04.117790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:04.118578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] RBAC: Adding query filter params (group.id=ea263ee495164034ad36a888f006ca88, role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:04.118887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:04.120350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:04.128201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 227/452] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:09:03 2022] GET /identity/v3/role_assignments?group.id=ea263ee495164034ad36a888f006ca88&scope.project.id=d02c4dd209ae4d3abdbc8e841c368c4b&role.id=cce34632ed81427db845cc21f77efac0 => generated 611 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:04.172433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bb9534-bacf-4e8d-8210-dd7f6dfe8e2c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:04.187204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:04.211303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:04.211906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:04.212188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:04.212462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] PATH_INFO: `/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:04.213713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] RBAC: Authorizing `identity:get_project(project_id=0e21ca24dfa94435a41c5ab2bd838cb9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:04.215164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:04.217090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 226/453] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:09:04 2022] GET /identity/v3/projects/0e21ca24dfa94435a41c5ab2bd838cb9 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:04.222745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:04.322615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:04.326761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d099aa-2107-4d6d-81c2-18a41d9dae6c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:04.360856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:04.361499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:04.361779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:04.362053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] PATH_INFO: `/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:04.363194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] RBAC: Authorizing `identity:get_group(group_id=6ea8d6adbb6540e39d4bfdc25ea25da1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:04.364764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225f27b0-69e5-4f17-bcce-9d9cfbc58bc3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:04.366533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 228/454] 10.222.0.40 () {56 vars in 1180 bytes} [Sun Jul 31 14:09:04 2022] GET /identity/v3/groups/6ea8d6adbb6540e39d4bfdc25ea25da1 => generated 229 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:04.372252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:04.476197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1893b742-66ad-4ebe-bec6-fa99235c07d8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:04.510394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:04.511006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:04.511283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:04.511585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:04.512378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] RBAC: Adding query filter params (group.id=6ea8d6adbb6540e39d4bfdc25ea25da1, role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=0e21ca24dfa94435a41c5ab2bd838cb9) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:04.512687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:04.514101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1e1ff4-1e37-43a2-a490-a14c9eb819d7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:04.520209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 227/455] 10.222.0.40 () {56 vars in 1376 bytes} [Sun Jul 31 14:09:04 2022] GET /identity/v3/role_assignments?group.id=6ea8d6adbb6540e39d4bfdc25ea25da1&scope.project.id=0e21ca24dfa94435a41c5ab2bd838cb9&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:06.870391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249d5bd-5bb1-4b1f-97fe-0eabb6125118 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:06.871446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249d5bd-5bb1-4b1f-97fe-0eabb6125118 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:06.871937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1249d5bd-5bb1-4b1f-97fe-0eabb6125118 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:06.873704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 229/456] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:06.880730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c685-8dc8-49e2-a247-5f479aa09f00 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:06.881541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c685-8dc8-49e2-a247-5f479aa09f00 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:06.882015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2497c685-8dc8-49e2-a247-5f479aa09f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:06.912386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-2497c685-8dc8-49e2-a247-5f479aa09f00 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:06.913896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2497c685-8dc8-49e2-a247-5f479aa09f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:06.963319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:06.998104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 228/457] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:07.033662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249d5bd-5bb1-4b1f-97fe-0eabb6125118 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.102144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fccee-bde2-49f0-b5e2-3c9ea65c3059 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.102463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fccee-bde2-49f0-b5e2-3c9ea65c3059 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.102921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fccee-bde2-49f0-b5e2-3c9ea65c3059 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.104340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 229/458] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:07.109224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.109498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.109770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.135873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:07.136979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:07.193325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1249d5bd-5bb1-4b1f-97fe-0eabb6125118 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:07.250885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:07.251551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.251830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.252099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.264295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 230/459] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:07.273346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:07.275078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:07.296693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:07.297565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:07.298406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:07.298406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:07.298406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:07.298406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:07.298406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 230/460] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:07.303845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.339069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.367778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:07.479623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e12423-c969-4f9d-b9ea-75bde7b378e6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:07.485469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:07.507226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44730355-cb3e-433e-a12e-24fd72b49998 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:07.546342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:07.546962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.547244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.547541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.548243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:07.548544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:07.549896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b659a93-b8c5-49c6-bbe3-796bc0de248a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:07.560424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:07.561422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.561904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.562393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.563059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 231/461] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:07.567547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f925-0001-4e57-a56c-6dddb06cd5c3 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.569512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f925-0001-4e57-a56c-6dddb06cd5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.569785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d467f925-0001-4e57-a56c-6dddb06cd5c3 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.571525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 232/462] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:07.575990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f925-0001-4e57-a56c-6dddb06cd5c3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.586902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:07.589332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-41179513-6319-4810-8d1e-d757ac027093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:07.607328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:07.608422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:07.609280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:07.609280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:07.609280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:07.609280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:07.609280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 231/463] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:07.610035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d92-fee5-41e6-8a3b-415dad024f06 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.610035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d92-fee5-41e6-8a3b-415dad024f06 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.610300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63085d92-fee5-41e6-8a3b-415dad024f06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.632196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-63085d92-fee5-41e6-8a3b-415dad024f06 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:07.633432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-63085d92-fee5-41e6-8a3b-415dad024f06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:07.691624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d467f925-0001-4e57-a56c-6dddb06cd5c3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:07.702542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:07.715565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 232/464] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:07.717771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10120-3fbd-4300-a332-cdb2123e4b40 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.718377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10120-3fbd-4300-a332-cdb2123e4b40 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.718935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b10120-3fbd-4300-a332-cdb2123e4b40 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.721115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 233/465] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:07.722261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b10120-3fbd-4300-a332-cdb2123e4b40 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.730428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:07.731172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.731486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.731766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.732451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:07.733877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a416ad7-192e-4ec5-8b4d-d46e04e03b0b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:07.763462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 233/466] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:07 2022] POST /identity/v3/users => generated 246 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:07.763462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.763462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.763462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.793500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:07.795145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:07.817144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:07.858809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b10120-3fbd-4300-a332-cdb2123e4b40 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:07.904098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:07.904758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:07.905013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:07.905269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:07.905986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:07.906292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:07.907727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:07.916505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 234/467] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:07.916505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:07.929248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 234/468] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:07.934265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.003812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.044047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c621f28-a6bb-4164-8ef9-8db3fd6571be None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.057471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea556d3-c25b-457f-901b-fd375e22ac0b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.084402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.085229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.085605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.085984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.097032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.097333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.098085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.098540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.098954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.099282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.100136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.102237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:08.115867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:08.116776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:08.117626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:08.117626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:08.117626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:08.117626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:08.117626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 235/469] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:08.118936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.123729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.136788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 235/470] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:07 2022] POST /identity/v3/users => generated 248 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:09:08.144934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.279357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf13b836-cde5-4576-b5ac-5a76ab3730ee None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.299707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.327182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.328177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.328669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.330209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.331134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:08.333976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.336843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.358570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 236/471] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:08.363967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.392908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c224f9-ce16-4589-9b04-b8d32bb13241 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.433264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.434017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.434362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.434703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.447281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.448872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:08.464229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:08.465132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:08.465974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:08.465974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:08.465974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:08.465974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:08.465974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 236/472] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 348 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:08.471409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.494483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6aad10-c2b5-4eab-9279-441589ab2146 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.536589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.544419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.545482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.546200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.546645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.548101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.549791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73366531-63e6-4223-92ae-bed70bdb5cb2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.581627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 237/473] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:08 2022] POST /identity/v3/users => generated 248 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:09:08.585051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea0ef-d380-4967-b2d3-f6c74987b854 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.585193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea0ef-d380-4967-b2d3-f6c74987b854 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.587817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ea0ef-d380-4967-b2d3-f6c74987b854 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.587817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 238/474] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:08.592568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.592870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.593132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.604181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29883d1f-8e27-45a3-b40f-05be856cb74a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.613578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:08.614858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:08.646215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.651460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.652688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.653155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.653648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.654810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:08.655917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.657614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.673956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 237/475] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:08.682766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.759252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 239/476] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:09:08.800871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:08.833307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee77e56-cf07-4cc9-aca1-e45efe4decd3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.851900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:08.881331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.882749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.883177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.883906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.884760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.887558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.919309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 238/477] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:08 2022] POST /identity/v3/users => generated 249 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:08.925434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffacc3a-908b-4b77-bec4-30dee296cce4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:08.970405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:08.971282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:08.971664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:08.972042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:08.972895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:08.974538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:08.983617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.sql.core [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 14:09:08.983617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 14:09:08.983617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [parameters: {'id': 'ce8d2a86a56f4e78a5327f40d35573dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 14:09:08.983617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80120) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jul 31 14:09:09.001824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return ref.to_dict() Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:09:09.002762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application yield resource Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise self.value Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application session.commit() Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 14:09:09.003714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.flush() Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application compat.raise_( Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 14:09:09.004593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:09:09.005580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:09:09.006383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [parameters: {'id': 'ce8d2a86a56f4e78a5327f40d35573dd', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:09.007184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 31 14:09:09.009989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:09.013612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 240/478] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 31 14:09:08 2022] POST /identity/v3/roles => generated 144 bytes in 203 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:09.013754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:09.135290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9566463b-a88b-40ae-9a05-42547fab0379 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:09.171492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:09.172176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:09.172459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:09.172735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:09.185643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d66a3b36-0ecf-43cf-b235-f1b8ad903a60 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:09.191052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:09.191977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:09.193137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 239/479] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:09.200026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:09.228601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:09.307666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe55449-d1be-44c9-bf17-9dfabd6e1930 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:09.342666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:09.343293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:09.343604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:09.343879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:09.344609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:09.344917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:09.346359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067daf4-7820-4768-aa0a-0ecd036a4a3b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:09.355435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 241/480] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:10.190225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa317-b7dc-4d62-9860-43b6b1438223 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.190765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa317-b7dc-4d62-9860-43b6b1438223 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.190765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4aa317-b7dc-4d62-9860-43b6b1438223 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.193030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 240/481] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:10.198625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.199048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.199221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.217348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:10.218540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:10.302686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 242/482] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:10.346180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa317-b7dc-4d62-9860-43b6b1438223 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:10.382466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:10.468162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4aa317-b7dc-4d62-9860-43b6b1438223 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:10.513443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:10.514099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.514381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.514705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.528458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-5e149cad-9bc0-4a00-b5f5-107098bde17e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:10.533383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:10.534234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:10.535061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 241/483] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:10.542232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:10.574904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:10.691503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6055a795-2425-4b53-b8ed-4926e0f211e6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:10.739813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96032b-7631-4a85-badd-bcf76df8acc2 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.740090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96032b-7631-4a85-badd-bcf76df8acc2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.740276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96032b-7631-4a85-badd-bcf76df8acc2 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.741859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 242/484] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:10.746765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.747044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.747319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.749560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:10.750187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.750471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.750747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:10.751496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:10.751810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:10.753173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:10.767770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 243/485] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:10.772797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:10.774410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:10.775608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:10.914324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 243/486] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:10.927560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:10.953093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d159250-6f78-46a0-846e-85ecf18e593d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:10.997185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:10.997748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:10.998026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:10.998298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.001169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.017144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.018762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.025805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:11.033971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:11.035037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:11.036582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:11.036582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:11.036582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:11.036582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:11.043451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 244/487] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:11.043451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.173390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0221085-079d-44b1-b2fd-062aa1645455 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.229452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c58f96-999f-4989-9524-9736d03609f9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.241367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.242628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.243089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.243594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.265063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:11.270799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:11.271814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:11.272594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:11.274666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 244/488] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:11.281352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.297760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.309239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.309877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.310162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.310439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.311156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:11.311476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.313093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783d7e33-f629-4211-806a-66279dbf831f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.334791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 245/489] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:11.336633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b8b81-2261-43fe-a398-dcb6fd8e1aef None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.336992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b8b81-2261-43fe-a398-dcb6fd8e1aef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.337341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b8b81-2261-43fe-a398-dcb6fd8e1aef None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.339458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 246/490] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:11.341193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b8b81-2261-43fe-a398-dcb6fd8e1aef None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.392896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.405499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1293f30d-2593-435e-a493-01188b4ca85e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.444357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.445346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.445613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.445890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.445994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b8b81-2261-43fe-a398-dcb6fd8e1aef None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.446617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:11.446922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.448598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796182c8-68be-47e3-b81e-bfe14f3cf5c2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.461572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 245/491] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:11.463026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.463316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.463703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.481128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.481773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.482051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.482322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.482538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:11.483757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:11.489743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.491161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:11.505172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:11.506120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:11.507484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:11.511111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 247/492] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:11.512006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.559652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 246/493] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:11.560624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.576457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.619858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ca994-9e50-48ea-8575-5028c17f87b6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.639773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.654284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.654921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.655200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.655500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.667861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.669492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.672910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aceeb8a5-e44e-4b17-a656-1397f2a84950 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9f3844ed-6cdd-4e17-b1b6-d9325d3dfd66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:11.684227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:11.685237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:11.686240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:11.686240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:11.686240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:11.686240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:11.686240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 248/494] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:11.686514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cce46-93f2-48a2-9c7e-45aa17508251 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.686791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cce46-93f2-48a2-9c7e-45aa17508251 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.687062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3cce46-93f2-48a2-9c7e-45aa17508251 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.691838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 249/495] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:11.693914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cce46-93f2-48a2-9c7e-45aa17508251 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.715971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.716587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.716864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.717437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.718042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:11.718347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.720176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.764495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 247/496] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:11.764495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.804039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3cce46-93f2-48a2-9c7e-45aa17508251 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.806697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.839938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.840589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.840948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.841294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.851844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bea86300-5fcf-4fb8-94e2-bdb5d2823a67 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:11.857205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:11.858268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:11.859273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:11.860577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 250/497] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:11.861590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.861952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.862289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.877550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f4c7177-4168-4599-8f0f-38e445fff155 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:11.880895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:11.882144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:11.907638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:11.914622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:11.915245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:11.915559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:11.915839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:11.916546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:11.916847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:11.918544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:11.934674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 248/498] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:11.934674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:11.958705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 251/499] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:11.959387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.048223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfee02b-b8ba-4455-bc3e-387318f2e930 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.057724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.066012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc3c8cb-05af-4b4b-b150-53db079e4859 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.085447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.086069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.086350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.086627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.100678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.101237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.101515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.101789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.102511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:12.102814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.104221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.116528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 252/500] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:12.116528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.145959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.156765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.158248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.216912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81695e-958f-4163-bd83-5cdd5956193c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.249329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.249953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.250229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.250531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.254047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 249/501] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:12.255402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.258401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.260003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0ba72bfb-b6b5-4225-ba5d-323cc9cbca81 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:12.271547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:12.272636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:12.273740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 253/502] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/users/glance => generated 84 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:12.274423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d88cc-58b0-492b-8b0b-b9141abe34e0 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.274423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d88cc-58b0-492b-8b0b-b9141abe34e0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.274423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d88cc-58b0-492b-8b0b-b9141abe34e0 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.275595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 254/503] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:12.280421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d88cc-58b0-492b-8b0b-b9141abe34e0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.298368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.349299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.359397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a0f691-fbf2-45cc-8ece-aae0f3232fd4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.382926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d88cc-58b0-492b-8b0b-b9141abe34e0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.392302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.392931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.393208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.393477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.403379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:12.408197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:12.409162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:12.409947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 250/504] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:12.410634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.420847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.421540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.422093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.422394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.432962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.434392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:12.449324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:12.450234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:12.451061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:12.451061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:12.451061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:12.451061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:12.451061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 255/505] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:12.451324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.510011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21d2b36-af82-4a35-b6e8-cb1d30ae362f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.513929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.544199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.544839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.545125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.545403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.547192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e52060a-0bce-4ea8-a75c-c552aae74ece None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.555788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.557234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7af25ab1-629f-4138-8929-c5a1a0e60ea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:12.575867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:12.576888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:12.577890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:12.577890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:12.577890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:12.577890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:12.577890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 251/506] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:12.578464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.578683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.578964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.584279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.584899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.585179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.585453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.586256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:12.586562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.588069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.588655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.601202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:12.602410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:12.624812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 256/507] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:12.625631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.673663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 252/508] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:12.674496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.725101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665cd45f-a3c1-4e78-abf8-ef4d0e557354 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.750466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.759557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.760165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.760441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.760709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.761417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:12.761713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.763056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.771366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 257/509] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:12.772073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.777521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb5d79c-3651-40b3-bdc7-14283d12fed8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.807436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:12.812436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.813044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.813322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.813591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.814281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:12.814579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.815961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.827784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 253/510] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:12.828653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.871575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4412986d-db56-4e5d-b5b7-3e8596081fd4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.904167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.904708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.904982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.905253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.905934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:12.906239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:12.907619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:12.918777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 258/511] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:12.919587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:12.929379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8cb1ea3-eba4-4fdd-b3b7-cce884a76716 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:12.961723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:12.962341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:12.962626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:12.962900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:12.968556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.018935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.019398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5632cd-3469-4253-8707-69c3e7c03c8f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.029523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.031171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.052955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.053593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.053874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.054147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.063917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.065343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:13.079970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:13.080980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 259/512] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:13.081923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.135061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 254/513] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:13.136084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.181903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.185372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37298a0-a5ea-4224-90f0-e3b5fd9df468 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.219299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.219956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.220232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.220504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.226944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.231391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.232851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.236350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d3e027-76ff-4970-8b2b-84054e062946 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:13.246783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:13.247728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:13.248623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:13.248623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:13.248623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.248623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:13.248623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 260/514] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:13.248880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.273586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.274227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.274510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.274785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.281806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.283364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:13.296563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:13.297644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 255/515] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:13.298846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.348792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e6c96-4720-4297-999a-0989a1a09446 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.381639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.382270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.382551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.382822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.389296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.390734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.396447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ddb9fd-69c2-4184-b90c-c4a6ed26911d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:13.402835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:13.403740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:13.404577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 261/516] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:13.405049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.419221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.436050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.436598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.436877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.437149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.437835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:13.438138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.439532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.451801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 256/517] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:13.452603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.466282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.506997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50362e2d-93a3-48a1-a8d8-b6a6fc383cf9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.539859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.540486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.540763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.541032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.548184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64e815c-1b57-4d47-87ad-4e1aab97134a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.551042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.553064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:13.566625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:13.567578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:13.568441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:13.568441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:13.568441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:13.568441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:13.568441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 262/518] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:13.568699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.585474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.586097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.586376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.586659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.587359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:13.587699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.589068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.604759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 257/519] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:13.604759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.633872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.669402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.671805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098f9c92-cec5-4f24-9529-ddb0be868cd4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.698696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ac7412-7651-4125-af3f-9381d8706735 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.705841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.706448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.706723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.706993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.707814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:13.708118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.709543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.726611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 263/520] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:13.727637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.735942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.736558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.736838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.737112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.737859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:13.738162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.739720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.757622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 258/521] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:13.758421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.826918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c28c0-aeb1-4dab-af5d-549d3f468037 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.848755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.856179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece79228-b1db-4a66-85e0-b52c694876a4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:13.860808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.861424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.861709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.861991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.862668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.864233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.884383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 264/522] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:13 2022] POST /identity/v3/users => generated 247 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:13.885183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:13.887109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:13.892743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:13.893389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:13.893672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:13.893951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:13.894696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:13.895077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:13.896528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:13.913911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 259/523] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:13.913911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.030553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e4a17f-a0a9-40ba-a142-b1165a5df778 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.030748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646acf46-621e-4cab-991b-77fd510e7330 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.070687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.071310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.071626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.071899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.074741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.075602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.075892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.077009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.087134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.088876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:14.103256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:14.104529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:14.105427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 265/524] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:14.105993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.146762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.153451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.157292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.158709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.217292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b385c04-93b9-4b4b-8145-a97c81b5a0c1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.252442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.253056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.253334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.253607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.254502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:14.254816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.256270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.271455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 266/525] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:14.271455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.280191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 260/526] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:14.281365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.404613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.406684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307717f8-764f-47d5-bff7-85edc32e9f03 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.408917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.417912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90274b21-3ce6-48a8-8adf-cd986433c2e8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.455826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.456666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.457038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.457447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.458013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.458635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.458917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.459193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.463457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:14.463457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.463457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.464723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.466164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:14.478308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:14.479223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:14.480143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 267/527] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:14.480601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.491510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 261/528] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:14.492399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.605883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc72907-488c-49a2-b27b-4b6072b0fa19 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.628884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d38a4a-e9e8-4f0a-83e2-88ab56c6a845 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.645188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.645917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.646270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.646623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.647479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:14.647860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.649400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.667815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 262/529] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:14.668733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.670972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.671906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.672226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.673408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.682530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.689232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.691165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:14.714663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:14.715828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:14.716860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:14.716860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:14.716860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:14.716860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:14.723864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 268/530] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:14.724710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.747694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:14.791078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce8068c-6ba7-4700-ad6f-91faf951dd71 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.833313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.833942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.834372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.834790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.863291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0792f4-9b76-4551-ad8d-0790c2e29009 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:14.911700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:14.912703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:14.913243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:14.913946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:14.915564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=42b59618e59540d1bcd5c2397b004422) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:14.916112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.917298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:14.918263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.918735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:14.925997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 269/531] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/role_assignments?user.id=42b59618e59540d1bcd5c2397b004422&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:14.927769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:14.932590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.093988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.117720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 263/532] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:15.122654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac40e6f2-07d8-4000-afb5-2099d7bfb0f2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.127571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.193647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.194271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.194556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.195089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.195825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:15.196133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.197512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.222522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 270/533] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:15.226519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.302227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f38116-55d5-4eae-a974-8d853e6d258c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.321178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.356434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.357108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.357388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.357663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.388508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.389969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.405935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee080ae-f1f7-450c-b5a7-7e613979eaed None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:15.421762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:15.422644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:15.423502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:15.423502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:15.423502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:15.423502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:15.427158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 264/534] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:15.432379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.450263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.450911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.452005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.452005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.464876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.467556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.472715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:15.490303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:15.492413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:15.494471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:15.503054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 271/535] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:15.503054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.634331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfcc7ab-d0dc-4b09-a48e-b49a1a79db89 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.670680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.671228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.671551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.671828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.672532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:15.672836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.674721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.688159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 265/536] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:15.694838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.727866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.734923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc147b6c-f6d7-4134-a230-620797d5a9f2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.810696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60b46ff-e2eb-4d70-8796-79a8507655a9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:15.828128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.828766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.829049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.829329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.830095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:15.830403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.831862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.845963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:15.846611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:15.846893 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:15.847166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:15.854861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:15.856408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:15.870776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:15.872817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:15.874967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 266/537] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:15.876092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 272/538] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 371 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:15.880471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.882611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:15.985881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:15.989895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4464267f-7ae7-4e25-8df1-d17333e93432 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.033672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.034484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.034768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.035061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.036913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:16.037227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:16.038674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bb1c09-565d-4adf-9770-15b954524c20 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:16.054960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 267/539] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:16.056352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fc4e7-7bc3-42df-b025-3ef1e912cefb None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.056638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fc4e7-7bc3-42df-b025-3ef1e912cefb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.056916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7fc4e7-7bc3-42df-b025-3ef1e912cefb None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.058372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 268/540] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:16.061616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fc4e7-7bc3-42df-b025-3ef1e912cefb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:16.069088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:16.117521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad64e129-1117-4406-af68-c3dd3419180f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.164258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.165148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.165435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.166000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.167547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=5bd24147c8c44ff7a9800bc406d51953) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:16.168137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:16.170226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d09935-0f07-4d76-97c0-ca22c24c0352 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:16.173502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7fc4e7-7bc3-42df-b025-3ef1e912cefb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.177911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 273/541] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:15 2022] GET /identity/v3/role_assignments?user.id=5bd24147c8c44ff7a9800bc406d51953&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:16.179111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75972e6-0aad-41f6-86f5-cf441802af7f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.179400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75972e6-0aad-41f6-86f5-cf441802af7f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.179705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75972e6-0aad-41f6-86f5-cf441802af7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.209628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b75972e6-0aad-41f6-86f5-cf441802af7f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:16.210891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b75972e6-0aad-41f6-86f5-cf441802af7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:16.255197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.255791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.256075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.256353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.257205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=a3c98dfd1f934a98b26889cda2d5c762) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:16.257522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:16.267253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:16.282592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 269/542] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/role_assignments?user.id=a3c98dfd1f934a98b26889cda2d5c762&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:16.351665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 274/543] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:16.380443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:16.397250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:16.496869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef301e-3da1-412c-a59f-953840f7f517 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.497403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef301e-3da1-412c-a59f-953840f7f517 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.497403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef301e-3da1-412c-a59f-953840f7f517 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.498860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 275/544] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:16.504795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.504795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.504795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.524719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:16.525981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:16.533721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a665bd-7e78-4d95-8f69-0d3a26c7a04f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.604840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.605506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.605830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.606117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.621438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:16.626619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:16.632649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:16.635112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:16.637545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 270/545] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:16.644953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:16.671694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 276/546] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:09:16.694224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:16.747828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:16.814848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42858830-fbba-4aad-855d-0f89911a7734 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.850808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.851808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.852294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.852794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.853796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:16.854309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:16.855977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:16.866910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 271/547] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:16.869536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bc145b-64df-4c83-b828-4808caef80da None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:16.872018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:16.948738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:16.949407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:16.949694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:16.949969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:16.967524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:16.979708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:16.980643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:16.981427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 277/548] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:16.989171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.013344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c5d82a-b57c-44b4-b40c-1da02f9c1bae None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.044035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.076265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.094651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.095301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.095611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.095884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.118877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.121153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.136277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176ee26e-eb59-45bd-b6c1-5dc4ecfbfc9b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:17.151377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:17.152442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:17.154192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:17.154192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:17.154192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:17.154192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:17.156275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 272/549] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:17.160850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.192463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.193090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.193373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.193644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.194435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:17.194996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.196505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffc4557-f5c9-466e-b390-b5f4820d3e4a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.205288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 278/550] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:17.206703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e755b3-8def-4a97-a2ca-cd273bceae3f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.206994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e755b3-8def-4a97-a2ca-cd273bceae3f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.207311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e755b3-8def-4a97-a2ca-cd273bceae3f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.208862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 279/551] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:17.214626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e755b3-8def-4a97-a2ca-cd273bceae3f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.289768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a1c18c-76c1-410e-a12c-ec3d39058b26 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.331151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.331345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e755b3-8def-4a97-a2ca-cd273bceae3f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.332035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.332921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.333480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.335075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:17.335503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.336973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-391a512a-8bbe-4ccd-8165-2862ca389bcc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.339255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.352256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 273/552] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:17.353649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.353992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.354347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.372030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.372910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.373394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.373845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.380714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:17.382562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:17.386083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.387835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:17.405385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:17.406251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:17.407219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:17.407219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:17.407219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:17.407219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:17.409253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 280/553] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:17.410324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.425015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.475472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 274/554] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:17.475472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.544359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1652a975-24d0-47e3-ab59-45f846ca0792 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.582925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d68978-c36b-4f27-ba3d-b4915be5ba17 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.584072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.584720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.584997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.585269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.593991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.595515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:17.608279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:17.609268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:17.610188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:17.614332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 281/555] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:17.615341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.621607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.622251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.622535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.622812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.623579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:17.623892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.625302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.636674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.638559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 275/556] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:17.639273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.688000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.722625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cb1a04-0d2d-4b91-aad4-32512e9cef92 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.752440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa33a2f-f7a0-451f-8848-7213a346e9db None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.757176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.758091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.758644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.759113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.769839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:17.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:17.776187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:17.777499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:17.780217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 282/557] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:17.781290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.790638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.791443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.791749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.792035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.793013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:17.793381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.795068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.816247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 276/558] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:17.816247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.868889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.893649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d5d3ff-a695-4057-844d-91091ec15c05 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.929254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.930277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.930754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.931224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.935759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:17.939742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.941440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:17.947993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad833f3d-958f-4a25-b92e-eeb35148d16a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:17.954239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:17.955248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:17.956195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:17.959536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 283/559] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:17.960680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:17.991725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:17.992397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:17.992713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:17.993212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:17.993961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:17.994270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:17.996364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.007632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 277/560] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:18.008487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.080900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd91a06-310f-4e82-8fc6-c3388f9f9def None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.123184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ea0ce8-0f48-4c84-9aee-ba19fb170b94 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.130444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.131068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.131359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.131657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.137179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.181626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.183156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.183666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.184116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.185490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:18.186390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.188367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.213070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 278/561] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:18.213252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.214269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.215085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.228566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.331133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baba2afd-5496-4489-9823-9cf02660cb48 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.348799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 284/562] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:18.349922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.373536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.374180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.374525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.375057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.385329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.387010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.396043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:18.403651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:18.404545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:18.405419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:18.405419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:18.405419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.405419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:18.409852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 279/563] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:18.411345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.473485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.488602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5b7772-6502-414d-8979-8a1e207824f3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.525931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6959ae-5350-48f6-801e-fee5adcae162 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.526829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.527616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.527987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.528355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.540986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.542903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:18.557703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:18.559080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 285/564] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:18.560020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.567770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.568408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.568689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.568960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.581040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.583252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-1cd74ef2-db77-468c-a37b-83e92ee644a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:18.597282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:18.598356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:18.599252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:18.599252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:18.599252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:18.599252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:18.599252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 280/565] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:18.604226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ef69-7ada-4b36-b8eb-3e85ac4f2488 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.604511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ef69-7ada-4b36-b8eb-3e85ac4f2488 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.604781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d2ef69-7ada-4b36-b8eb-3e85ac4f2488 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.606024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 281/566] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:18 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:18.607021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3963-a9fc-432f-8fc3-9209c7226089 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.607357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3963-a9fc-432f-8fc3-9209c7226089 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.607653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240d3963-a9fc-432f-8fc3-9209c7226089 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.611455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 282/567] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:18.611455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3963-a9fc-432f-8fc3-9209c7226089 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.657378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.671138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ec11c4-636e-42cb-9c4d-30af2a98ec1a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.704893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.705558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.705838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.706107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.706791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:18.707089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.708985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.716889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240d3963-a9fc-432f-8fc3-9209c7226089 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.720308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 286/568] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:18.721089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.730312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.751448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.752056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.752337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.752610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.753303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:18.753607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.754985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cb1dd9-0742-48d4-82db-008ba8287c35 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.766146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 283/569] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:18.767153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b010-6a26-45f8-87a0-990f202b6155 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.767504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b010-6a26-45f8-87a0-990f202b6155 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.767733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd1b010-6a26-45f8-87a0-990f202b6155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.786525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-edd1b010-6a26-45f8-87a0-990f202b6155 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:18.787767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-edd1b010-6a26-45f8-87a0-990f202b6155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:18.820614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4868972a-e8d5-4821-b537-b54ebca5e0fc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.852628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 284/570] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:18.854119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.854406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.854620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:18.854754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.855238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:18.855548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:18.855828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:18.856520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:18.856826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:18.858193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:18.873948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 287/571] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:18.873948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.875198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:18.876417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:18.884486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:18.942702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 285/572] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:18.943544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:18.972875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-495fb64e-ee9e-4e94-8a4e-b408bca16440 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:18.976689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.006084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.006748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.007032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.007304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.016296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.017772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:19.032490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.033657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:19.035634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:19.038441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 288/573] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.039346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.068258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf817b6e-b7e4-4ed3-85f2-20984e7135a6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.102347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.102991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.103265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.103574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.109966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.110115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.111411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:19.122720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.123650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:19.124559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 286/574] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.125046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.148068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d3d2bd-dec9-49d3-9b29-6b0ab343ccd6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.180095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.180793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.181072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.181344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.191016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:19.197076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.198002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 289/575] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.198885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.225615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.234816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55607126-6f65-4f12-9b1b-79a3a1403257 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.267551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.268176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.268452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.268719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.275704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.277112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:19.288898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.289852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:19.290744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 287/576] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.291258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.300567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5488f310-02c5-4311-91b8-8a6401a7ab21 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.327351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.336518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.337167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.337446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.337726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.347900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:19.353493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.354494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:19.355328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 290/577] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.356277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.404824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e238db-b1b3-44b2-980d-46d0bcedf73a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.447413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.447970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.448250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.448524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.449332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:19.449640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.451103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.460366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e038af5-da21-47c3-ab05-116cadab2612 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.467751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 288/578] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:19.468550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.481130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.495751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.496361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.496636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.496909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.497646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:19.497945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.499355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.511712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 291/579] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:19.512500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.559576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.575507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56e790da-049e-4764-a010-e07bfb272bea None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.607937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.608545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.608674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e476e7-494e-41ac-9997-98201217d468 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.608809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.609076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.609780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:19.610079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.611414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.619529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 289/580] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:19.620302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.645587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.646207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.646489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.646767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.647548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:19.647859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.649281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.660285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 292/581] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:19.661059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.691382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.720855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af182469-e7d7-4315-86c4-8d90d52629af None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.758674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.759288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.759595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.759866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.760580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:19.760886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.762230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.769941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 290/582] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:19.770706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.777838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b9f401c-cc73-4385-abe5-79e42f6d5c13 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.788355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.814509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.815157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.815460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.815743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.826208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.827676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:19.841270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:19.842147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:19.843029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:19.843029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:19.843029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:19.843029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:19.843029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 293/583] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:19.843301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.872357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1210e39-26b2-4c69-98ea-8cacb7af6866 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.905134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.905788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.906075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.906351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.908795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:19.922027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=d04ab058ac79408f80cc485082c6591c, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.923477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:19.936164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 291/584] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:19 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:19.936947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:19.940002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b96121e-971b-4214-a138-1092ec4b88fd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:19.976546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:19.977438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:19.977869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:19.978290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:19.991300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:19.992882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:20.008535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:20.009575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:20.010465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:20.010465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:20.010465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:20.010465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:20.014670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 294/585] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:20.015648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.032262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.059545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b4905e-01e3-4c3d-85ed-318a1eb649fa None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.114443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.115069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.115345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.115761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.116572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=9d498354314244afa4445a1ed99d15cf) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.116935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.118457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.126158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 292/586] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:19 2022] GET /identity/v3/role_assignments?user.id=9d498354314244afa4445a1ed99d15cf&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:20.127096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.149633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789e08f0-83d8-4a8c-8a97-edec6dcbc211 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.188982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.203096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.203950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.204347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.204624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.221138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.224036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:20.245244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:20.246248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:20.247146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:20.247146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:20.247146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:20.247146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:20.247146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 295/587] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:20.251567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.256196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dc831b-9c26-455e-9684-de3ed56b17a6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.307745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.308671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.308953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.309380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.310394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.310714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.312331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.328601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 293/588] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:20.329161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.330435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.371871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ab4540-9dfc-40ac-9a7d-5a24ed42acfb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.415039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.415693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.415976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.416255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.416947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.417252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.418644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.433340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 296/589] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:20.433340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.457880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128e3d36-becb-4ee3-a358-9d058347c871 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.495617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.496665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.497245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.497774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.498678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.499182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.500891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.514084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.517889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 294/590] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:20.519043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.541257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f3d9b6-f72c-42b7-a151-0bd080c856ec None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.567097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.579889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.580594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.580902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.581368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.588466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.590054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:20.603890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:20.604864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:20.605777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 297/591] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:20.606260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.633166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9464590c-9594-4815-9618-0c2d4b0f0941 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.673395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.674356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.674822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.675303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.685999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.687861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:20.700586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:20.701754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:20.702916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:20.709730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 295/592] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/users/cinder => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:20.710855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.736990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55538a87-6a1f-42b7-b0b7-69c4570923b4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.745400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.783796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.785896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.786439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.787192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.797289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.799751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:20.817439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:20.818400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:20.819352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:20.822837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 298/593] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:20.823862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.828102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3df816-75e1-445c-9868-a31ceae005b2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.866262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.866888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.867167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.867458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.868283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.868588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.870119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:20.885026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 296/594] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:20.885997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.886111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:20.953707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52444e7b-0e2d-4a4c-9541-b46dd679db6c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.978840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:20.990804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:20.991663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:20.991861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb61a16-b528-4fc9-aa74-a9413a798ad7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:20.992260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:20.992726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:20.993809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:20.994304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:20.995886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 299/595] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:21.015544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.031592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.032524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.032524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.032744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.033553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:21.033979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.035894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.057845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 297/596] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 31 14:09:20 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:21.058740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.125747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e477c32-ff05-4e0b-a2b7-0ce77900b8e3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.159024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:21.163211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.163920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.164203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.164631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/42b59618e59540d1bcd5c2397b004422/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.171535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88ec9de-aa34-45e5-b493-84703b51ac2d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.185093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=42b59618e59540d1bcd5c2397b004422, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.187478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.207733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 300/597] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:21 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/42b59618e59540d1bcd5c2397b004422/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:21.209020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.221557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.222673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.223171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.223725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.248184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.250767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.269207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:21.287550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:21.288561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:21.289429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:21.289429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:21.289429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:21.289429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:21.293329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 298/598] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:21.295363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.429664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750d54d9-5711-44c3-aa6c-ff1f933fddf9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.461808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624fa0b1-8088-4799-993f-4e9d86fc9ed1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.478871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.480377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.480974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.481446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.498023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.500734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.505804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.506427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.506706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.506980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.507844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:21.508153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.509506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:21.527640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:21.528549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:21.529441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:21.529441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:21.529441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:21.529441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:21.529441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 301/599] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:21.531189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.535771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:21.542049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 299/600] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:21.548274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.644315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:21.691291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a910bcf7-1447-4140-96d5-54f9a3256356 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.730920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35e9bb0-d20a-4071-9260-39d8aeaf44a5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:21.770577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.771289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.771592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.771865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.772570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:21.772875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.774853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.808205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 302/601] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:21.814273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.819053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:21.820889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:21.821174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:21.821447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:21.835652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:21.837843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:21.860006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:21.860937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:21.861803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:21.862495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 300/602] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 314 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:21.867173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:21.937270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:21.990020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261b8c76-ec8d-40fa-89c4-e12db6258f34 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.045917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.046407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.046694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.046970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.054780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.056251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.057116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1800fd7-8c19-4232-b6ae-65a4fc6f4be4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-91b1e725-f06b-46a8-96f6-6c4aa9cb5177 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:22.068782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:22.069961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:22.070879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 303/603] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:22.071871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12fcb3-93e0-4acd-9ab5-062625befcbd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.072039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12fcb3-93e0-4acd-9ab5-062625befcbd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.072311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a12fcb3-93e0-4acd-9ab5-062625befcbd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.077731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 304/604] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:22 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:22.078687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12fcb3-93e0-4acd-9ab5-062625befcbd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.080015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.104814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.106341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.106728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.107107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.108130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:22.108598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.110187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109207ed-4471-4994-a5db-4cae3fa4097e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.125129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 301/605] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:22.126503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.126890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.127266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.146959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:22.148360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:22.197287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a12fcb3-93e0-4acd-9ab5-062625befcbd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.227444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 302/606] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:22.227444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.234464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.252941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f688c-7967-41d5-961f-f2483996062e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.254343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f688c-7967-41d5-961f-f2483996062e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.254856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f688c-7967-41d5-961f-f2483996062e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.255335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f688c-7967-41d5-961f-f2483996062e None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.256875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f688c-7967-41d5-961f-f2483996062e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:22.257396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f688c-7967-41d5-961f-f2483996062e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.259658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f688c-7967-41d5-961f-f2483996062e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.277357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 305/607] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:22.278442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f688c-7967-41d5-961f-f2483996062e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.342261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfdad82-a4d2-4872-82ac-6da00ba31c12 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.352917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.380343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.381023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.381304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.381572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/5bd24147c8c44ff7a9800bc406d51953/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.396559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f688c-7967-41d5-961f-f2483996062e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.398844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=5bd24147c8c44ff7a9800bc406d51953, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.400376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.415583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 303/608] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:22 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/5bd24147c8c44ff7a9800bc406d51953/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:22.416379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.439229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.440367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.440890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.441340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.454509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-95aaae6d-4d55-4421-9161-fa9656969a11 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:22.460500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:22.461573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:22.462414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:22.466731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 306/609] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:22.468335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716121-ba2e-4cde-aeba-483821738570 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.468675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716121-ba2e-4cde-aeba-483821738570 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.468955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81716121-ba2e-4cde-aeba-483821738570 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.470445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 307/610] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:22.471173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716121-ba2e-4cde-aeba-483821738570 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.542822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.575478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4444e8d9-ca4c-4d76-aeb4-17e4e8ee2bc2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.617956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81716121-ba2e-4cde-aeba-483821738570 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.622145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.622957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.623244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.623635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/a3c98dfd1f934a98b26889cda2d5c762/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.648343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=a3c98dfd1f934a98b26889cda2d5c762, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.649783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532bdce4-6dcf-4f5f-af3d-84030d31647f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.663941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.664860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.665293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.665757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.666680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:22.667145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.667855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 304/611] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:22 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/a3c98dfd1f934a98b26889cda2d5c762/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:22.668974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.669359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.669747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.670097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.682753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 308/612] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:22.687935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.699234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:22.701210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:22.704774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.823218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232fd6df-6c35-4fac-b280-d202c759b489 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:22.831264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:22.842474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 305/613] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:22.862520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:22.863521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:22.863989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:22.864424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:22.876387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:22.876866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:22.878548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:22.895470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:22.896425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:22.897746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:22.897746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:22.897746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:22.897746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:22.897746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 309/614] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:22.902692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.036766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51baed8b-bb1c-4c01-bff2-a44be07cafce None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.076484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.077385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.077667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.078340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.085562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054c012f-680b-4850-8b53-6adde1ce4b9e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.086354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.091113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:23.097329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:23.098474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:23.099523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:23.103021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 306/615] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:23.110525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.196327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.196327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.196605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.196845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.199441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:23.199441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.199441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.225033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 310/616] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 323 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:23.225905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.261640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.273437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1f4aab-92ae-48f3-bcbd-736113f41759 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.329192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.329818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.330099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.330372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.331099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:23.331404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.332831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.358132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 307/617] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:23.360605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.400365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6a9181-d812-4c73-9208-568618671d29 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.455282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.456949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.457233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.457506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.470390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.471856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.478599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:23.496614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:23.497701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:23.498653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 311/618] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:23.507862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.557975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dce8182-1ba2-42f9-9bb0-6e5d520fb63b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.593954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.605468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.606113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.606397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.606673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.618184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabd68a-1804-40e4-ad04-f4a4395c29f1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.621792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.623243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a2afa371-080c-4f19-a5cf-dbb90f562b0e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:23.641316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:23.642235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:23.643264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:23.643264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:23.643264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:23.643264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:23.647219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 308/619] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:23.648833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379f2e-9beb-445b-aea0-641167df439d None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.649122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379f2e-9beb-445b-aea0-641167df439d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.649398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb379f2e-9beb-445b-aea0-641167df439d None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.651703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 309/620] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:23.652714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379f2e-9beb-445b-aea0-641167df439d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.662074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.663018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.663577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.664466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.665542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:23.666362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.668652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e135a1ad-5aa1-4f0a-bcfd-51c31f250ad1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.695661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 312/621] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:23.697890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.697890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.698194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.715337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:23.716574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:23.769206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb379f2e-9beb-445b-aea0-641167df439d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.791445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 313/622] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:23.791445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.799767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.829299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.829943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.830234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.830516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.831240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:23.831590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:23.833566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:23.846315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 310/623] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:23.847272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:23.875856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:23.913028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565f7e74-5d72-4e1d-8851-43d48110dece None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.951964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43efc621-4efb-4fda-aa13-c909cb0e2993 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:23.955524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.956261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.956643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.957566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:23.988394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:23.989183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:23.989465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:23.989742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.001456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:24.007471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.008561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:24.009389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.013936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 311/624] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.014916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.043327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.045246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.075811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.123562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88673dae-10d3-4a35-86ff-d8107bc883f3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.124726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.172256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.173285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.173783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.174305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.176106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 314/625] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:24.177024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.184106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.185904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:24.199347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.200329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:24.201288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.207749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 312/626] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.208921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.288517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d832e9-6fdd-442c-841b-056c807d1d9d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.323001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.327690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.328548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.329178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.329659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.330690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:24.331214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.331444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6534eae-eb4c-437d-b2b4-06875840e551 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.333874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.348353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 315/627] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:24.349547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.372557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.373203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.373484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.373759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.382229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.387793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.389248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:24.407295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.408387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.409269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:24.409269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:24.409269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.409269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.409269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 313/628] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.410454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.469551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b5b99-5a2a-434f-b7bb-0fb77f2dbb0e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.508030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.509092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.509602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.510091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.511089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:24.511725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.513526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.520796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330ef8ae-08d2-4cd8-9a56-fe82d24aad7e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.533460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 316/629] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:24.539150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.557842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.558486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.558768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.559044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.570675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.572194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.574833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:24.587997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.588951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.589853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:24.589853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:24.589853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.589853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.596391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 314/630] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.596391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.650366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.669233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701fba3e-974d-4017-9cf2-c9d97ace6172 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.700773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7549afb9-c11b-4e91-bee0-8ca9db7004f8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.708545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.709419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.710218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.711928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.714427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:24.715089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.718019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.734503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 317/631] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:24.735965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.736552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.737193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.737475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.737746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.749616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.751071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:24.766010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.766954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.767880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:24.767880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:24.767880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:24.767880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.772360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 315/632] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.773226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.826100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.850313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d69f8f-bb98-4b56-a0a7-8f591e77a8f5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.881506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761be1b8-2643-48ef-97c7-fb11e0da457b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:24.885914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.886538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.886816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.887092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.887815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:24.888124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.889482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.907059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 318/633] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:24.907962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:24.917239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:24.917426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:24.917886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.918168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.918441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.925317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:24.926766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-0eef5a59-40fe-412b-9edd-181be8f6c2bf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:24.937756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:24.938663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:24.939571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 316/634] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:24.940098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3862f8-fd9e-4319-bd5e-95802ff7764f None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:24.940382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3862f8-fd9e-4319-bd5e-95802ff7764f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:24.940650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3862f8-fd9e-4319-bd5e-95802ff7764f None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:24.942390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 317/635] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:24.943138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3862f8-fd9e-4319-bd5e-95802ff7764f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.015867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e0a6d7-24da-4af3-a362-d8c11374ac21 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.041548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3862f8-fd9e-4319-bd5e-95802ff7764f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.049246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.049864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.050143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.050412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.050871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.051128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.051455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.052801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa0fb7b-6121-401a-b16c-6314f15b9fcf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.064033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 319/636] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.065345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55567-41da-4392-8082-2599485bd6e8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.065634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55567-41da-4392-8082-2599485bd6e8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.065908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb55567-41da-4392-8082-2599485bd6e8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.067675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 320/637] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:25.068369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55567-41da-4392-8082-2599485bd6e8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.082460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.083094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.083370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.083678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.090572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.092039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-49bfe9c9-b037-4a71-99e0-4fd124430598 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:25.103184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:25.104263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:25.105216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 318/638] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:24 2022] GET /identity/v3/users/glance => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:25.106344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.106344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.106344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.122619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:25.123818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:25.140050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.177775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb55567-41da-4392-8082-2599485bd6e8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.189193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 319/639] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:25.190023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.214678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.215297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.215599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.215874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.216620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.216926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.218350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86682176-a782-4072-b737-6cbc0dc35085 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.230716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 321/640] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.231912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.232202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.232478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.249602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:25.250797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:25.284335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.288332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc8c913-7292-47cf-97c2-27746436a9b4 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.315070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 322/641] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:25.315918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.335222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.335891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.336169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.336437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.343403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.344830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:25.355685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:25.356676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 320/642] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:25.357998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.373715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.416929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3584940e-a0e6-471d-ba68-dc559804b6cb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.450653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.451262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.451575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.451847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.452636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.452943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.454365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.458258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c60726-e742-4784-bbe7-f3c6010dee51 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.474913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 323/643] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.475835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.492152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.492790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.493069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.493339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.503251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:25.509104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:25.510210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 321/644] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:25.511315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.532250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.578328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac3a3a4-19c8-4dcf-adeb-89f642814c42 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.591636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.610947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8a1d45-e0df-4a2f-95dc-2755e666937a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.611370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.612011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.612295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.612570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.613879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.614188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.615632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.621396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 324/645] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.622163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.648161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.648836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.649115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.649393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.660889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:25.666382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:25.667584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:25.668458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 322/646] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:25.669152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.740774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757c65b1-dadc-4041-b793-4be20f66475e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.760881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.786545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96477f8d-0447-43b0-9de9-6b8c97791b9b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.790319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.791123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.791531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.791874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.792662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.792954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.794386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.808008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 325/647] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.808915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:25.825119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.825746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.826029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.826306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.856388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:25.907082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.908607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.921653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f8bac92-f685-4f7b-be24-0ae8067fe34b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:25.957706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:25.958317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:25.958602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:25.958877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:25.959631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:25.960033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:25.961305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:25.969512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 326/648] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:25.975178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.080795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.085161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7674fc-8c32-4a03-850b-ce34f2b10d9a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.107673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.121424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.124889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.125174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.125452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.126181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:26.126487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.127884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.139196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 323/649] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 467 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:26.139196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.139486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 327/650] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:26.139486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.247578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8751eff-e3ad-425c-9a85-027f6ce6e0a0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.254023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a6e008-18c0-4518-affe-a5cd0c6d7212 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.283189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.283882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.284159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.284427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/9d498354314244afa4445a1ed99d15cf/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.288235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.288873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.289153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.289434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.299858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.300763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=9d498354314244afa4445a1ed99d15cf, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.301332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.302587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:26.323448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:26.324462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 328/652] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:26.325410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.325773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 324/652] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:26 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/9d498354314244afa4445a1ed99d15cf/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:26.325773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.348321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.360139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.467130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580d7eb-1e87-4a02-bac0-c742feaa9292 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.476467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c682fc-aabb-4517-81f6-8b7665a71937 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.507136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.507806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.508084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.508932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.515266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.515936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.516216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.516485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.521564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.523015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.528319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.530445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:26.538402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:26.539285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 329/653] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:26.540557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:26.552468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:26.553409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 325/654] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:26.555786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.630599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.640785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.661915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb73624a-e90d-497e-bf42-8ffe1a521904 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.681173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3120a2-e7c2-46eb-baf7-5d2a2d6bac1d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.706513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.707829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.708173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.708516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.709833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:26.710206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.712154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.723875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.724558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.724843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.725118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.725818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:26.726181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.726962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 326/655] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:26.727626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.727842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.747566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 330/656] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:26.748563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.864894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db46f21d-dbc8-4104-a456-d7a4cc64afb0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.865669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5fb7ab-e7e7-4d8a-8f5b-a705b69d13ab None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:26.899726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.904125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.905289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.905658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.905947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.907850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:26.908703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:26.909295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:26.909575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:26.910305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:26.910607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.912002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.913947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:26.916066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:26.922290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:26.931150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 327/657] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:26.932034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:26.932591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:26.933815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:26.934713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:26.938024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 331/658] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:26.939065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.041363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e1389e-2582-412d-b48d-5f1eefe5d403 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.044081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5f9fca-ffc5-48e7-b918-9256c9e92c51 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.077625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.078449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.078571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.078829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.084084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.085058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.085553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.086065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.086442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.087912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.094610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.096366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:27.099497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:27.100471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 31 14:09:27.101444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:27.105233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 332/659] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:27.106084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:27.113571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:27.114543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:09:27.115604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:27.118052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 328/660] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:26 2022] GET /identity/v3/users/cinder => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:27.119053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.168200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.190129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.223860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896c3c55-eca1-47fe-84ae-9d37fe049913 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.236800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2239e585-8284-4a4c-a0ab-a4f2d098a443 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.258305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.258956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.259233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.259532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.260284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:27.260615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.262014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.274507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 333/661] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:27.275494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.276498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.277356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.277938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.278430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.279594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:27.280129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.281817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.307168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 329/662] 10.222.0.40 () {56 vars in 1139 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:27.308515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.386900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ceab5e2-0c2b-4daa-8394-58250df25952 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.395467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.426895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.427928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.428471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.428947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.429961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:27.430477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.432152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.441704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68feae1c-1faf-4fe9-b728-cc8b151d1058 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.453832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 334/663] 10.222.0.40 () {56 vars in 1137 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:27.456230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.465554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.485322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.486335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.486825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.487333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.488978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=42b59618e59540d1bcd5c2397b004422) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:27.489496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.491174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.497861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 330/664] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/role_assignments?user.id=42b59618e59540d1bcd5c2397b004422&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:27.498888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.596360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b63826-79c2-4217-b1d9-a694a06446dd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.624521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3dbd6a-82d5-4f4a-9ea5-0d6d0001e63f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:27.653494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.654558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.659943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.660574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.678544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:27.679093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.679372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.679668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.727078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.782158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:27.788347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.790460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0878308b-71ba-4c1c-b85f-165e3d03c066 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.810100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:27.811658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:27.906674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 335/665] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:27.908012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aae7d-552a-429d-894c-7ffe15ec07c1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.908300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aae7d-552a-429d-894c-7ffe15ec07c1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.908573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19aae7d-552a-429d-894c-7ffe15ec07c1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.910105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 336/666] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:27.917256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:27.917545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:27.917819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:27.929829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 331/667] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:27.929829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:27.940773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:27.942650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:28.032944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.037018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.051651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 337/668] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:28.052545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.087979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64ffe82-c252-4049-8e9e-151915c278c8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.153549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.154207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.154491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.154767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.169150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.170608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:28.186968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.188144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.189020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:28.189020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:28.189020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.189020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:28.193095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 332/669] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.193614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.212217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5deaf119-8f87-45f2-b348-168568a12014 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.257465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.259018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.259525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.259977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.273355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.275737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:28.294522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.295461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.296361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:28.296361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:28.296361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.296361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:28.298198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 338/670] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.298198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.309470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c3c4ed-f279-40ad-a2fe-4bfc9f5332e1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.346879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.348479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.348981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.349569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.351902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.355932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.369909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:28.375987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.376934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:28.377850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:28.380748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 333/671] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.381862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.416541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb26cd2-95d2-4865-b162-5f737d78d2df None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.453857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.454502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.454788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.455179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.455907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:28.456213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.458232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.477125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 339/672] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:28.477125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.520940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965cfea6-6279-4bbd-8f61-472defd49f84 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.566116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.567454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.568180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.568619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.569980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:28.570494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.572422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.585539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 334/673] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:28.587754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.590752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6de75f-e545-48ca-9d1e-bd583eb34622 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.598883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.625515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.626138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.626421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.626696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.627414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:28.627747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.629155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc1d23d-a382-43bf-9877-8600bb351a9e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.637125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 340/674] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:28.638273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b7db-99ef-474a-a062-8ab4ed5725f5 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.638557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b7db-99ef-474a-a062-8ab4ed5725f5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.638832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a3b7db-99ef-474a-a062-8ab4ed5725f5 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.640211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 341/675] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:28 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:28.640744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3b7db-99ef-474a-a062-8ab4ed5725f5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.651342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.697445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6813542c-cb93-436d-bff8-a1463855e25f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.733131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.733962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.734242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.734516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.741175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.742032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a3b7db-99ef-474a-a062-8ab4ed5725f5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.742600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:28.754167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.755054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:28.756000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 335/676] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.756473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.784550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.785223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.785500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.785835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.795115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.796608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-17f2d4ad-3607-4473-8e23-9ed9c5dd2664 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:28.808000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.808896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:28.809776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 342/677] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.810749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.810749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.810991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.829069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:28.830249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:28.853929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.864836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de07855-b76a-429b-a0e5-0283992c5e21 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:28.879143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:28.900641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:28.901314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:28.901561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:28.901919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:28.905804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 343/678] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:28.906827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:28.913187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:28.914719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:28.930636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:28.932080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 336/679] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:28.932982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.021405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c186970-7822-4244-b6ac-61e70deb6203 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.039292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83121c7-725d-4ff4-ac6d-0c7db0a88769 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.056537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.057390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.057828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.058250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.059130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.059626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.061039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.078182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 344/680] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.078182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.080197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.080821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.081101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.081376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.082704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.083011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.084466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.096779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.102134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 337/681] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.102910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.133456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.189147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fc98a8-8c16-4a80-9961-795077a5345a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.220374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e083c307-97c3-46c8-81c0-6f1007dea695 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.225135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.225779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.226061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.226336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.237685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:29.244194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:29.245326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:29.246184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 345/682] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:29.246905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.262789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.263410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.263742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.263992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.264685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.264989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.266344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.278298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 338/683] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.279096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.338477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.360829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64fdb49-b67e-4e63-9c0c-ac0b0d269ea1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.372136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.389603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee7bc59-f3b0-45dc-9ae4-19eb03636c26 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.397961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.398577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.398849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.399121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.400016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=a3c98dfd1f934a98b26889cda2d5c762) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.400333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.401744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.407678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 346/684] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/role_assignments?user.id=a3c98dfd1f934a98b26889cda2d5c762&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.408453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.437892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.438848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.439325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.439757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.440693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=5bd24147c8c44ff7a9800bc406d51953) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.441685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.443295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.450650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 339/685] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/role_assignments?user.id=5bd24147c8c44ff7a9800bc406d51953&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.451723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.557806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfac2e-36cb-446a-bb74-bd7543588feb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.583474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c75584-3b08-44a2-847d-8f06251d81e2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.606441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.607172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.607592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.607974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.608816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.609243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.610770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.621368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 347/686] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.631474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.638757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.639311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.639628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.639821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.639944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.650592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.653357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:29.668684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:29.669709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:29.670638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:29.673759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 340/687] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:29.681969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.702633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.799198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64e780-5fca-4d78-872c-cb690f65cfeb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.815740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a65b3-39bf-494b-a569-d7aeb468c147 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:29.836493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.837146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.837427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.837804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.850069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.851453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.853089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:29.853832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:29.854217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:29.854498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:29.855316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:29.855658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:29.857091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:29.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:29.867731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:29.868606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:29.868606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:29.868606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:29.868606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:29.868606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 348/688] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:29.877402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.885327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 341/689] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:29.891565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:29.948199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:29.981977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.000536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf691e2d-809d-446f-8362-99c4c83a98e9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.029890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ef268e-26c3-4f8b-9adb-3b44e1cf61ce None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.036772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.037915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.038369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.038791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.039778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:30.040231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.041876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9ae2d8-736e-4b50-badb-6b3ac9f54914 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.054965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 349/690] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:30.056554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed67b3-3b9f-4e83-a2da-9ec6def3151a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.057049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed67b3-3b9f-4e83-a2da-9ec6def3151a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.057539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed67b3-3b9f-4e83-a2da-9ec6def3151a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.059409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 350/691] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:30.060347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed67b3-3b9f-4e83-a2da-9ec6def3151a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.077399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.077971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.078250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.078524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.162928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.164731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.190327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed67b3-3b9f-4e83-a2da-9ec6def3151a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.210704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.235108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.235765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.236038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.236300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.238470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.246601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.247406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-542e78b8-78d1-4e22-9941-bf1dfcfdf6d1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:30.259847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:30.260817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:30.261793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 351/692] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:30.262899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.262899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.263073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.281529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:30.282714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:30.289004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 342/693] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:30.290077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.377925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 352/694] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:30.379813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.417121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a56392e-12b1-436b-ab25-a4ad2176b98b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.456385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.457930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.458440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.458919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.460763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:30.461280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.463464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.487602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 343/695] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:30.489340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.504476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6ce332-3782-46ea-a506-94b43773851e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.514824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.529117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.548665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.549716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.550193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.550641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.561857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.563836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:30.580525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:30.581417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:30.582907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:30.582907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:30.582907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:30.582907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:30.584858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 353/696] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:30.585811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.626830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fa4025-8bf8-4498-8645-7ad5b7f53242 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.670640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.671722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.672007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.672285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.686288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.688038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.702005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38d75bc-9c09-4546-b029-2993bc23ed27 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:30.705949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:30.706890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:30.707871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:30.707871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:30.707871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:30.707871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:30.714600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 344/697] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:30.715662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.743643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.744668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.745025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.745397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.781957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.805945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:30.837177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.839070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.839789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac42de0-1708-42bb-9c70-076c193b6faa None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:30.876297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:30.877303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:30.877779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:30.878316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:30.879852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:30.880448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:30.882163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:30.896374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 345/698] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:30.897602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:30.967465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 354/699] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:30.968760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.026646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e68fd93-201e-4ce7-975a-2972ab133891 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.032814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.070563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.071188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.071493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.071774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.072648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:31.072956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.074988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.082497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7582640-7fa6-4bb8-8401-7c00cccd8d28 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.083272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.093834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 346/700] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:31.093834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.132471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.133469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.133945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.134414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.142197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.147175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:31.161643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:31.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:31.163556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:31.170538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 355/701] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:31.171733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.207585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f253ea-68b3-48dd-8e77-4e64e3ff3f80 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.261756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.262337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.262620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.262895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.275030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.276945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.290392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c27fa8-98cf-4705-8aef-01d6fa4fd541 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:31.293460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:31.294336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:31.295230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:31.295230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:31.295230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:31.295230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:31.302276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 347/702] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:31.303602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.310587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.332338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.332965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.333245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.333520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.334379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.336493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.366683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.375814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 356/703] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:09:31 2022] POST /identity/v3/users => generated 254 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:31.377999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.472145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5dc198-8564-448e-b847-ded89a050935 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.548828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.549866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.550883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.551555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.553842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:31.554561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.557738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.580414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 348/704] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:31.580637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95416f31-2d8a-444e-80d9-0db1df0ebda7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.594097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.638411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.638760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.638950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.639225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.639971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:31.640279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.641657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.657194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 357/705] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:31.666809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.704954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.729655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03566d43-f8db-47bb-ab73-4869cf0587c8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.742482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:31.770984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.772119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.772572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.773320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.774260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:31.774715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.776890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325d8a9f-6517-4f9f-ab07-2a798c12b446 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.786285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 349/706] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:31.788604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838ffb-8c40-40cc-a3c4-faf86e721195 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.789547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838ffb-8c40-40cc-a3c4-faf86e721195 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.789974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9838ffb-8c40-40cc-a3c4-faf86e721195 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.791380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef46dd8-b18d-4450-9b3e-0a6363bb86fe None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:31.792129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 350/707] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:31.798404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079abf-4b8f-4f50-b899-8539825b9b11 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.798723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079abf-4b8f-4f50-b899-8539825b9b11 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.799009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22079abf-4b8f-4f50-b899-8539825b9b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.824385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-22079abf-4b8f-4f50-b899-8539825b9b11 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:31.826387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-22079abf-4b8f-4f50-b899-8539825b9b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:31.850733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:31.852554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.853080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.853878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.865791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:31.867973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ca87a4b5-cd7b-4e7c-9743-5be59b27066e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:31.880654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:31.881662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:31.883059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 358/708] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:31.887615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a734dd-0f3d-45eb-a39a-4e598de275ed None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.887995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a734dd-0f3d-45eb-a39a-4e598de275ed None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.888278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a734dd-0f3d-45eb-a39a-4e598de275ed None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.890019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 359/709] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:31 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:31.891603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a734dd-0f3d-45eb-a39a-4e598de275ed None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:31.928912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 351/710] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:31.929110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1870-b47d-4069-9826-440ba926863a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:31.929367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1870-b47d-4069-9826-440ba926863a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:31.931994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92aa1870-b47d-4069-9826-440ba926863a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:31.958040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-92aa1870-b47d-4069-9826-440ba926863a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:31.959217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-92aa1870-b47d-4069-9826-440ba926863a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:32.015240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:32.025432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a734dd-0f3d-45eb-a39a-4e598de275ed None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.044967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 352/711] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:32.046603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1870-b47d-4069-9826-440ba926863a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.079184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.080256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.080762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.081258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.082754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.083715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.085362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.088764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:32.100583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 360/712] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:32.101856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.189064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92aa1870-b47d-4069-9826-440ba926863a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.216986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c2d162-b7d2-43ac-9c0d-929c6461a2ca None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.240571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.241843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.242583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.243542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.244733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.247298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.254040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.254686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.254960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.255230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] PATH_INFO: `/v3/services/network` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.255900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.257242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:32.262394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:32.263374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:32.264255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:32.268140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:32.269665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:32.270500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:32.271222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 353/713] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:32.271222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.271364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 361/714] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services/network => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:32.271945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.291175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:32.377870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8489e70-5b8a-428d-9254-50a10632d797 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.381811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:32.386746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdecd9a-f6e5-412e-83f5-cd1057957390 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.417792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.418714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.419193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.419700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.420563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.421228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.422807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.433355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 362/715] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:32.433355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.434126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.434723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.435464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.435867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.436669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=9d498354314244afa4445a1ed99d15cf) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.436984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.438673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.447564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 354/716] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/role_assignments?user.id=9d498354314244afa4445a1ed99d15cf&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:32.448396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.529849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:32.551884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ede312-c76d-4fb5-b64a-ae207043a8e1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.575468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261999ef-dcf8-46a5-99e8-a7b0f0149cb6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.591092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.592219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.592713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.593204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.594127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] RBAC: Adding query filter params (name=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.594644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.596286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.603105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 363/717] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:32.611022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:32.617144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.617781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.618064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.618342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.619036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.619345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.627453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ba0377-72fc-4517-b823-7f41c9ef4049 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.629176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 355/718] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:32.776861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88b6f48-aa1f-49e8-8cf5-10abc95f5a92 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:32.822286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:32.822926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:32.823204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:32.823505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:32.824202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] RBAC: Adding query filter params (type=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:32.824503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:32.825905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8ba507-433d-40cb-b210-1874c8c76174 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:32.834119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 364/719] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:32 2022] GET /identity/v3/services?type=network => generated 124 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:33.808849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe164f-1ad5-44e2-bb73-0bfe97f834f2 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:33.809370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe164f-1ad5-44e2-bb73-0bfe97f834f2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:33.809474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe164f-1ad5-44e2-bb73-0bfe97f834f2 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:33.810984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 356/720] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:33.817091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809153df-64b1-4674-bf77-1349f89b0d4e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:33.818013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809153df-64b1-4674-bf77-1349f89b0d4e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:33.818454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809153df-64b1-4674-bf77-1349f89b0d4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:33.839794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-809153df-64b1-4674-bf77-1349f89b0d4e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:33.841228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-809153df-64b1-4674-bf77-1349f89b0d4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:33.847666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:33.929606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 365/721] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:33.975723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe164f-1ad5-44e2-bb73-0bfe97f834f2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.005456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:34.100172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe164f-1ad5-44e2-bb73-0bfe97f834f2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.142995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a6a8-cf8d-45f4-9fdd-7287900cee4f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.143303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a6a8-cf8d-45f4-9fdd-7287900cee4f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.143601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a6a8-cf8d-45f4-9fdd-7287900cee4f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.147281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 366/722] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:34 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:34.151930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.152218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.152499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.161503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:34.162151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.162435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.162706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.183713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:34.184903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:34.187484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:34.192391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:34.193341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:34.195046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 357/723] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:34.200510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.392717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:34.423094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147a797c-71c5-43f4-8f8f-63a95b31e709 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.441319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 367/724] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 31 14:09:34.462096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:34.462789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.463170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.463532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.464317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:34.464684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:34.466127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:34.473070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.484615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 358/725] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:34.492065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.583210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:34.626816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25387d6e-7714-4d85-98f4-62e2b484d79a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.653469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce3470a-f81a-4c69-8b16-1b71a3a7b850 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.674438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:34.675318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.675631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.675900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.697398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:34.698925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:34.709530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:34.710174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.710451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.710721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.713983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:34.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:34.722002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:34.723386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:34.723386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:34.723386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:34.723386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:34.723386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 359/726] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:34.723692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:34.728103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:34.729005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:34.730372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:34.731188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.731326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 368/727] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:34.734833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:34.882448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dccd75f3-f704-4d47-8388-8fce35dfd8ba None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.920724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2398171b-bffd-41f6-925b-a6989086071d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:34.952822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:34.953462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:34.953739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:34.954009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:34.954717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:34.955016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:34.957336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:34.991838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:34.993446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 360/728] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:34.998007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.008371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.009024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.009309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.009588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.010335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:35.010645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.012156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.027773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 369/729] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:35.033289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.066735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.108625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb66ffc-4a39-498f-a054-c142334220cf None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.156159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.156830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.157074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.157346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.164991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.166426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.171317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71278fb-b9f0-46c2-adb8-d8ab59af8415 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-eb9408f1-b9f1-4863-b27c-1fe6b6f6947d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:35.178972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:35.179997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:35.181569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 361/730] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:35.182172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f290d9-704a-497e-ba06-f4bb44b0064d None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.182172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f290d9-704a-497e-ba06-f4bb44b0064d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.182515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f290d9-704a-497e-ba06-f4bb44b0064d None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.184257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 362/731] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:35.185476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f290d9-704a-497e-ba06-f4bb44b0064d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.221536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.222439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.222896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.223564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.238494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.240003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b3a94997-bd65-4e42-85a6-f613199bafdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:35.255794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:35.256707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:35.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:35.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:35.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:35.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:35.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 370/732] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:35.258519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.258741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.258937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.276961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:35.278147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:35.304670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f290d9-704a-497e-ba06-f4bb44b0064d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.322653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.331296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.350821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.351855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.352360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.352847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.353924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:35.354478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.356190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.356584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 371/733] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:35.357418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.376621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 363/734] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:35.388504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.468944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111fabe-8440-4b0f-a972-9d34f17f8953 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.494653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7586ce-4da8-4d7a-918e-1780764c8adc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.502649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.503304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.503670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.503951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.504643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:35.504946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.506314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.517395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 372/735] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:35.517997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.531952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.532571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.532850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.533118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.552930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.577940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.600899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.602330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9839df32-f2d4-439c-a16b-2f3552324638 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.617117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de168da9-ee88-4358-a1b8-29acb0ce8bc3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.648470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.649076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.649365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.649626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.650337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.651712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b2767d-aea9-4704-bcc6-8345663c8754 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.667610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 373/736] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:09:35 2022] POST /identity/v3/services => generated 251 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:35.668837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a68ed-48d8-49e2-9077-22565927230b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.669126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a68ed-48d8-49e2-9077-22565927230b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.669405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a68ed-48d8-49e2-9077-22565927230b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.670626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 374/737] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:35.671746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98d2c-7ae6-41d7-93f9-4df4cb40dd03 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.672028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98d2c-7ae6-41d7-93f9-4df4cb40dd03 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.672301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce98d2c-7ae6-41d7-93f9-4df4cb40dd03 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.673876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 375/738] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:35.674652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98d2c-7ae6-41d7-93f9-4df4cb40dd03 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.713658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 364/739] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 31 14:09:35.715220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65169a-7f5a-449b-93bf-bc2f164c11f1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.715687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65169a-7f5a-449b-93bf-bc2f164c11f1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.716108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a65169a-7f5a-449b-93bf-bc2f164c11f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.736239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9a65169a-7f5a-449b-93bf-bc2f164c11f1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:35.737476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9a65169a-7f5a-449b-93bf-bc2f164c11f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:35.803641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.823921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce98d2c-7ae6-41d7-93f9-4df4cb40dd03 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:35.826228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 365/740] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:35.827528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.827818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.828097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.837496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:35.857708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:35.859035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:35.870918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:35.871615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:35.871897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:35.872176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:35.880116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:35.881563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:35.892846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:35.894317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:35.895270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:35.895842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 376/741] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/users/nova => generated 82 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:35.896437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:35.936313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 366/742] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:35 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 31 14:09:35.937816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.021251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6db546-e83e-4d7d-a827-bf3b95de01e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.044089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942e26b-4096-4202-9865-a3a147280ab7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.058326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.058972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.059247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.059548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.072376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.073878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:36.088770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:36.089672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 377/743] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:36.091274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.094712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.095057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.096600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.097411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.097687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.099889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.109916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:36.114907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:36.116050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 367/744] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:36.122436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.205264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a464f40-1a58-4dd0-9d14-4073fc6eda92 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.232465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f0b38-3319-40ba-b44a-50f6f7d08766 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.240470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.241039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.241323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.241599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.242464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:36.242717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.244172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.268074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 378/745] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:36.268074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.273381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.273943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.274223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.274495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.275189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.276576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.293532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 368/746] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:09:36 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:36.294619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.355856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.365301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.410568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5317bf3-5b7b-4ad4-9dad-2550ca4741e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.436659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45e68f04-5544-45e0-8e30-07905629d265 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.463013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.464302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.464589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.464867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.465573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:36.465878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.467723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.482705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 379/747] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:36.483940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.491345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.492041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.492325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.492597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.493323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:36.493625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.494974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.506798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 369/748] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:36.507943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.618271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1feaed-1408-4556-bdbf-48e413fede76 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.622024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c02c1d-13e1-4e82-9d3a-5b22a0e4b211 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.636043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.646161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.662252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.663244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.663744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.664200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.665289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.665945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.666230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.666508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.672538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.674232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.680664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.682091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:36.692560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:36.693508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:36.694680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:36.698655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 370/749] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:36.699548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:36.702881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:36.703800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:36.704668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:36.704668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:36.704668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.704668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:36.710607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 380/750] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:36.710607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.817986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d56a41-bd50-474e-8d97-e52f23d45dad None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.851766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5acff54a-bc26-4793-be0a-eaddc4a2e4c9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:36.854049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.854761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.855179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.855456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.867939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.869587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:36.885525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:36.886903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:36.887839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:36.887839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:36.887839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:36.887839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:36.892097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 371/751] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:36.893739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.906504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:36.907852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:36.908343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:36.908808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:36.910128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:36.910991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:36.912966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:36.924300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:36.930072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 381/752] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:36.931405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:36.972295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:37.006573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f0ad457-1aef-4bbb-a4c4-195a1661b7d0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.041733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.042446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.042839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.043180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.044019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:37.044392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.045848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.063489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 372/753] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:37.063489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.070424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d315b8b2-425e-47fb-8d1e-0628107cb0f0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.105728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.106347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.106638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.106932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.107626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:37.107934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.109298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.126111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 382/754] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:37.127064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.186825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:37.207631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e5ce5b-e4eb-44f8-b62a-91d799256e8b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.232234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:37.243513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49dcba2f-1405-4848-bdc6-1cb5fc876eea None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.251334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.252273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.253146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.253635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.255100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=eb9910cbfb414432b3425c7dfdf79787) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:37.255922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.258100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.265715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 373/755] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/role_assignments?user.id=eb9910cbfb414432b3425c7dfdf79787&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:37.266761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.280432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.281269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.281614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.281955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.302005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.304622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:37.326624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:37.327604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:37.328497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:37.329380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 383/756] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:37.337684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.504634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a8d63-34ef-494d-b5c6-3ae1018750ac None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.545913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e02dc-a756-45d8-b1cc-0d6cb3aa11c5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.565628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.566642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.567813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.568464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.582565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.586947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:37.607651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:37.608606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:37.609553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:37.615352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.616269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.619040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.619567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.620547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:37.621062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.622612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 374/757] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 356 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:37.624217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.625077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.647779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:37.651601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 384/758] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:37.660574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.683527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:37.800563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ccb6cb-e0d3-4b42-bf15-4249d1ee1ff0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.832201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7130b4c6-6517-477f-b17b-be0e9e93519d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:37.840381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.841268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.841745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.842238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.843282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:37.843907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.845596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39d8f5-6b8c-459e-b8e6-68d33aee70d4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.865979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 375/759] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:37.871767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:37.871767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.871767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.871767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.872076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=d04ab058ac79408f80cc485082c6591c, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:37.872471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbafceb-53fb-46b7-9b20-dd27fdc9f3d1 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.872867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86224-1a21-49d5-98e4-0e7a8138c8cd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:37.873191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbafceb-53fb-46b7-9b20-dd27fdc9f3d1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.873661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbafceb-53fb-46b7-9b20-dd27fdc9f3d1 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.875481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 376/760] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:37.876407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbafceb-53fb-46b7-9b20-dd27fdc9f3d1 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:37.892629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 385/761] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:37 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:37.898499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c859569-e823-409e-85de-44162d2ef426 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:37.898499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c859569-e823-409e-85de-44162d2ef426 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:37.898499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c859569-e823-409e-85de-44162d2ef426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:37.931320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5c859569-e823-409e-85de-44162d2ef426 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:37.932579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5c859569-e823-409e-85de-44162d2ef426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:38.020441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.081652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 386/762] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:37 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:38.095469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbafceb-53fb-46b7-9b20-dd27fdc9f3d1 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.104535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c859569-e823-409e-85de-44162d2ef426 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.106518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.145086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.146008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.146431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.146870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.218941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c859569-e823-409e-85de-44162d2ef426 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.219643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.221228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.255842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.256595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.256976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.257887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.258086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.259593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:38.269175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:38.270327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:38.271335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.273515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 387/763] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:38.279248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.292850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.332853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 377/764] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:09:38.338682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.349286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.388770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b794917-7176-4f82-b83b-897c48af7798 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.426007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.426808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.427192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.427496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.428190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:38.428487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.429861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09716769-7e0f-4446-8b0d-abf35ea12df0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.435079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 388/765] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:38.440929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc49ba-788d-40d8-afab-750c2188921c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.441222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc49ba-788d-40d8-afab-750c2188921c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.441507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacc49ba-788d-40d8-afab-750c2188921c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.442991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 389/766] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:38.444501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc49ba-788d-40d8-afab-750c2188921c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.444949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af99938e-2d7c-443a-9491-5219db0fb8d0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.494427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.495894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.496483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.496993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.511944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.514185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-3f80c348-b3d0-41bf-80f2-64937b6dce12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:38.534748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:38.535715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:38.536605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:38.536605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:38.536605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:38.536605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:38.542244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 378/767] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:38.542244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.542244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.542244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.563446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:38.565220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:38.566213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacc49ba-788d-40d8-afab-750c2188921c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.584328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.597984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.605815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.606700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.607062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.607515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.608239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:38.608614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.610049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.616835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 390/768] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:38.617856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.656264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 379/769] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:38 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:38.657156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.740495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fe3b43-ea10-4408-8e12-6f7a59d5ab09 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.761527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f4f012-81d4-4439-bfd2-80a04e708fc7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.782814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.784031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.784431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.784750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.785580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:38.785869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.787473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.805368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.805581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 391/770] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:38.805581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.805729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.805729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.805729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.805729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] RBAC: Adding query filter params (interface=public, service_id=9b8d6db9677a49639f071b23ba0b3b9b, region_id=RegionOne) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:38.805729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.807057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.816026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 380/771] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/endpoints?service_id=9b8d6db9677a49639f071b23ba0b3b9b&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:38.817184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.834716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.872102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:38.921143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bdf23c-564e-4c72-83c5-2fe4d4476a26 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.936712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58175ad-aa7b-4df1-86c0-cd2e67360d3b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:38.959997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.960734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.961104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.961451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] PATH_INFO: `/v3/services/network` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.962256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.963820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:38.973715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:38.974587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:38.975039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:38.975508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:38.976867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:38.977845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:38.978680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:38.979409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 392/772] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/services/network => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:38.981256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:38.986274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:38.988436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:39.000196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:39.001262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:39.002134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:39.009721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 381/773] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:39.010751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.082041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.095521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efa94a7-2255-4dd8-a5b5-183c11ab5ac8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.113920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5985f9-d9df-468e-aa8e-98d5b3c402d9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.114562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.135016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.135801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.136157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.136676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.137306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.138672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.144571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 393/774] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:09:38 2022] GET /identity/v3/services => generated 772 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:39.145384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.161581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.162653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.163128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.163628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.164559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] RBAC: Adding query filter params (name=network) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:39.165056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.166616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.171955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 382/775] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:39.178563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.260015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e729c1-c082-496d-a041-95bb81959f2f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.294672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda03792-e328-460a-b967-775892e7fb26 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.299282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.300514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.300797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.301076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.301837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:39.302145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.303604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.318974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 394/776] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:39.327485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.356954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.358101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.359909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.360353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.361573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] RBAC: Adding query filter params (type=network) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:39.364087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.365648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.372549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.379415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 383/777] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/services?type=network => generated 350 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:39.385596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.423053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.498023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94973490-27d1-41da-b539-c3cb64b3f305 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.533524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad31f694-8649-4351-90ea-27efa9baf374 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.537940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.538721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.540093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.540521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.541461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=9d498354314244afa4445a1ed99d15cf) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:39.541934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.543511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.551408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 395/778] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/role_assignments?user.id=9d498354314244afa4445a1ed99d15cf&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:39.608350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.608923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.609283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.609568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.610442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] RBAC: Adding query filter params (interface=public, service_id=8a06da82a326487f99af111002704c6d, region_id=RegionOne) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:39.610756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.612304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e82888-7af7-4d67-82ef-ac5cdf8eec40 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.618474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 384/779] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/endpoints?service_id=8a06da82a326487f99af111002704c6d&interface=public®ion_id=RegionOne => generated 194 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:39.624398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.678031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b93da-44f0-4f42-a9a4-570270eab209 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.678319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b93da-44f0-4f42-a9a4-570270eab209 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.678603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1b93da-44f0-4f42-a9a4-570270eab209 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.680368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 385/780] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:39.685577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.685866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.686145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.721043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:39.722914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:39.755948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74bf910a-80f6-45b2-b9a3-f450ad668d07 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:39.765168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.801370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:39.802302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:39.802764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:39.803293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:39.804173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:39.805830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:39.811894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 396/781] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/services => generated 772 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:39.822508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:39.832639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 386/782] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:39.855993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:39.985955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fc6416-a4af-4a88-9316-a0b359387b8f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.025385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.026028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.026308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.026585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.037664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:40.043170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:40.044202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:40.045034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 397/783] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:40.050605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.160117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8354de-d9ba-4a11-abb9-62399547cb6f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.197119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.197749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.198024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.198297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.199017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:40.199318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:40.200704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b38600-1384-4672-a96e-61cb9539fb08 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:40.210247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 387/784] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:40.215700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.262940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:40.334299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d94a-a5c0-4150-8b88-9eae1f457a91 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.334648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d94a-a5c0-4150-8b88-9eae1f457a91 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.334858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b4d94a-a5c0-4150-8b88-9eae1f457a91 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.336501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 388/785] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:40.344776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.345063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.345349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.349117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dbfc0-52aa-4741-bdb0-305311a918a7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.384901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.385555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.385844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.386130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.387209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:40.388442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:40.396994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:40.398434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:40.413938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:40.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:40.416035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:40.416035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:40.416035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:40.416035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:40.416035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 398/786] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:40.422897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.480743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:40.508656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 389/787] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:40.546689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.582566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:40.614307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9415d44-c511-43cd-8505-6ac703c151f1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.677765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.678387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.678669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.678943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.680912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:40.681222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:40.682599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:40.709053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715cf552-8a3e-474b-98b6-064242d5870f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.711411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 399/788] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:40.712392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.777599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.778253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.778534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.778810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.798879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:40.804641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:40.805674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:40.806497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 390/789] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:40.812245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:40.892507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:40.908746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49499b4-f245-4cda-bf51-3932f45bc2f3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.977664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:40.979323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:40.979977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:40.980524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:40.989963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7fc9ea-dcbb-457c-97b0-2e2a3d964cd9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:40.999900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.001412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:41.021779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:41.022784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:41.023764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:41.026156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 400/790] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:41.029971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.037171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.059893 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.060444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.060726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.061002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.061739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:41.062044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.064680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.080302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 391/791] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:41.092870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.199657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07fc062b-1b3f-4aa3-9a6e-7a245fc44bc2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.237294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.237918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.238189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.238468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.239277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:41.239627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.241065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.260543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 401/792] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:41.265958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.333445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc9e4bcf-a712-4996-a63b-87b7d03e41b5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.347808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.355792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.379515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df328eb-f5a2-44b5-bc4d-0aa149f79ba5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.417564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.418283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.418571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.418852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.442894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.444671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.451546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.452203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.452488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.455804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:41.473701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:41.474650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:41.475561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:41.475561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:41.475561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:41.475561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:41.475561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 392/793] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 383 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:41.477383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.479053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.484490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:41.495180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:41.496096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:41.496999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:41.496999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:41.496999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:41.496999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:41.499154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 402/794] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:41.511645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.600688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d1c666-28fc-4653-81d7-70ec3b0ffd52 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.636265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.636886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.637170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.643915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.643915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:41.643915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.643915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.654482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 393/795] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:41.660068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.683862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.689745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e8a44-bde3-4fd4-96f9-3b752c9072a5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.721977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.770730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.771357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.771662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.771937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.772643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:41.773576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.775744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.781045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfdc0848-383a-42e7-a953-98656e47e946 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.801671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 403/796] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:41.807284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.837110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.837762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.838042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.838314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] PATH_INFO: `/v3/users/nova` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.846021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.847709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:41.860480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:41.861437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:41.863004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 394/797] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:41.872389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:41.947191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2441951f-a565-42de-9380-04fa34c6d449 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.960819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:41.980483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c705e2c3-08d9-4265-95dc-97cf5a4aaace None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:41.984681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:41.985529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:41.985959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:41.986387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:41.994425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:41.996297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e32773b3-3a3c-4b8c-b0c4-19370ac983fe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:42.009065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:42.010140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:42.011197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.017876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 404/798] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:42.017876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c77ee-edb7-4a82-983d-198b6643081a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.017876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c77ee-edb7-4a82-983d-198b6643081a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.017876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399c77ee-edb7-4a82-983d-198b6643081a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.018742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 405/799] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:42.019311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c77ee-edb7-4a82-983d-198b6643081a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.020328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.020951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.021230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.021501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.022478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:42.022641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.024075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a135642-272c-4c4c-86cc-cc82d92418b9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.047017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 395/800] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:09:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:42.047017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.047017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.047017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.063246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:42.063450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.064567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:42.138861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 396/801] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:42.142543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.147129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399c77ee-edb7-4a82-983d-198b6643081a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.181074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.181681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.181957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.182224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.182959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:42.183258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.184711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea6cf64-1a9f-4bdc-994e-3972ecb4fcc0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.197497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 406/802] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:42.198740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241b7a-681f-443a-a082-d865f81d87ef None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.199035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241b7a-681f-443a-a082-d865f81d87ef None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.199311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d241b7a-681f-443a-a082-d865f81d87ef None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.200558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 407/803] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:42.203882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f62e-3c6f-45d0-9e3e-d74831566241 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.203882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f62e-3c6f-45d0-9e3e-d74831566241 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.203882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e398f62e-3c6f-45d0-9e3e-d74831566241 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.203882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 408/804] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:42.204457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398f62e-3c6f-45d0-9e3e-d74831566241 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.214552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.248593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb7f320-df6d-4bec-a0a5-a706d20ec40f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.282196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.282812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.283094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.283382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.295645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.306469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e398f62e-3c6f-45d0-9e3e-d74831566241 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.339857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.340485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.340838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.341106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.341753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.343111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.350209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.351686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b129173a-8efd-43cc-ad29-449c91d9fdf9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:42.352137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.354472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 409/805] 10.222.0.40 () {56 vars in 1129 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:42.355406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.417633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.451936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 397/806] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:09:42.452674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782b8693-f1a0-419d-bce0-d22bce03daa7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.453192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.453481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.453763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.470640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:42.472342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:42.486693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.487926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.488203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.488476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/eb9910cbfb414432b3425c7dfdf79787/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.503519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=eb9910cbfb414432b3425c7dfdf79787, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.504956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5537f1d-bf26-443c-95ec-4fb857a4b11d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.517905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 410/807] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:42 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/eb9910cbfb414432b3425c7dfdf79787/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:42.519174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.519490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.519762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.533523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.541407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:42.542172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 398/808] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:42.543032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.549127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:42.640943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 411/809] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:42 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:42.641980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.660134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.665109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2595cf-9510-437f-b602-1d5ea9b6fa8a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.707682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.708510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.708796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.709072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.709958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:42.710569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.711967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.718612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 399/810] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:42.719775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.765721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adab9f81-2174-40f5-a682-8f44e024cf7f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.806104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.806801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.807083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.807355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.810426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.819753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.821210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.822213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ef988e-9bf6-4294-8212-dbd0f09a7d47 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:42.836370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:42.837282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:42.838212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:42.838212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:42.838212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:42.838212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.838212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 412/811] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:42.838487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.864606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.865589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.866054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.866510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.877499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:42.883325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:42.884441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:42.885494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:42.892215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 400/812] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:42.893176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:42.919555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:42.948272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb2886-3ffe-4e98-8364-1f8f4f0fc34f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:42.986233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:42.986797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:42.987109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:42.987381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] PATH_INFO: `/v3/services/network` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:42.988057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:42.989418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:42.998973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:42.999988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 31 14:09:43.000829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:43.001638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737181bd-e98c-4de0-a3db-487150c09d1b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.001721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 413/813] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/services/network => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:43.001721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.047331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.048266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.048546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.048818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.049533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.049841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.051206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.057162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 401/814] 10.222.0.40 () {56 vars in 1124 bytes} [Sun Jul 31 14:09:42 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.058216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.062497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.107611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c957ed-995e-4517-a4e7-8350fd415b7b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.146866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.147881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.148161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.148432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.149448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.149908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.151281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.156319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.163865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 414/815] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.168914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.174204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb63261b-68c7-44a7-9b12-35a19e6b3726 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.210069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.210785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.211139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.211519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.212255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.212557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.213923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.222534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 402/816] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.223306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.282106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe2c202-c482-4a35-b761-c63dc2afc268 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.296614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.320762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.321399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.321844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.322121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.323115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] RBAC: Adding query filter params (name=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.323461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.324834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.326145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13132d13-d5d9-4b01-b7e0-90f362fa51a4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.334306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 415/817] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.334306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.365210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.365852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.366136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.366409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.367116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.368517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.395472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 403/818] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:09:43 2022] POST /identity/v3/endpoints => generated 346 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:43.395472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.399448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.467078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5729fec1-61b5-4de2-ad32-f13282bbd7b2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.528447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ba648f-8186-4e57-82ef-26adaa84f7f5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.530681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.531372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.531673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.531948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.542575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.543991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:43.556685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:43.557678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:43.558574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:43.562919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 416/819] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:43.563834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.584854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.585507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.585785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.586057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.600160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.601591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.602501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:43.618298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:43.619263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:43.620229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:43.620229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:43.620229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:43.620229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:43.620449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 404/820] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:43.621149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.683817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.690990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2052fe-f95a-4107-9218-411256163ceb None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.727110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.727814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.728098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.728374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.729076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] RBAC: Adding query filter params (type=network) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.729511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.730713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.732843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e388147d-f2c1-4e58-9fd4-96c6f82af0ba None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.736152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 417/821] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/services?type=network => generated 350 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.736982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.770260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.771228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.771853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.772445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.773511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.774042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.775705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.789355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 405/822] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.790846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.837494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.843376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92a5d10-89fb-4f3d-a1b3-4ab4f4ac9f2e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.878480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.879152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.879456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.879734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.880427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:43.880733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.882440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.899538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 418/823] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:43.900631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:43.901879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65343725-ec52-4fdc-aaaf-159583a95f84 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:43.919836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:43.939609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:43.941194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:43.941709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:43.942197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:43.943113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:43.945028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:43.969958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 406/824] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:09:43 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:43.971920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.009696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1083e2f-2336-46ac-80f8-ace8cb3d6cde None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.055619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.056199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.056479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.056752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.057618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:44.057909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.059387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.066457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 419/825] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:44.106678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c626c6-94ee-4078-bb4a-c41f20a8024b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.157453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.158178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.158419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.159430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.172318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.173843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:44.196484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:44.197525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:44.198487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 407/826] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:44.206163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.233112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:44.331677 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d11404-39dd-43c9-8792-c59d1cb8d605 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.367565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.368001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.368277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.368554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.369356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:44.369655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.371077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.389004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 420/827] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:44.394750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.433024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:44.508353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d626eb71-f876-49c6-9a98-454ee153d892 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.543895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.544542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.544825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.545281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.556741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.558192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-e82b1da5-4272-4a3b-991d-1094a4edb49c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:44.573285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:44.574171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:44.575047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:44.575047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:44.575047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:44.575047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:44.575047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 408/828] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:44.581249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.592683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937f42f-10b7-45dc-a433-a681024eaf02 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.592982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937f42f-10b7-45dc-a433-a681024eaf02 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.593266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f937f42f-10b7-45dc-a433-a681024eaf02 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.594901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 409/829] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:44.601814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.602115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.602399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.625086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:44.626299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:44.665862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:44.692678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2774bdb2-0135-40c1-94c6-bafdb3fa2b42 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.700972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 410/830] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:44.729765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.742022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.742991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.744699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.745187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.746094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:44.747297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.749020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.753171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:44.764721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 421/831] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:44.772457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.879662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622dc494-7b6e-406d-9c77-7b74508fde8a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.883754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a056e95-1559-4238-996b-daa0bdd8f29d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:44.914398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.915049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.915334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.915656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.917864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:44.923746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:44.925740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:44.932465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:44.933587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:44.934056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:44.934532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:44.938603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:44.939507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:44.940975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:44.943685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 422/832] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:44.946922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:44.949949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:44.956681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:44.957644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:09:44.958520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:44.961587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 411/833] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:44.967674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.064715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3c6027-2f5c-440b-b930-4082a6165b09 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.071772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.100642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ae284e-8309-4e5a-beb0-24041cdeb6bd None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.112385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.112967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.113360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.113738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.114602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:45.115029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.116589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.135282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 423/834] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:45.147225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.154350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.155123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.155567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.155949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.156861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:45.157271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.159631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.172991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 412/835] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:09:44 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:45.174436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.221560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.282196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a525d611-c57d-4b8b-b632-55fc9c07d390 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.286276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2980eb-2422-4c0d-a455-5bb3bc7ff92f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.318301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.319401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.319827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.320164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.323501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.334077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.335897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.339293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.340584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.341091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.341580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/9d498354314244afa4445a1ed99d15cf/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.344941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=9d498354314244afa4445a1ed99d15cf, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.346980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:45.350917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:45.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:45.352720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:45.352720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:45.352720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:45.352720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:45.357002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 413/836] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:45.358298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.366489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 424/837] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:45 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/9d498354314244afa4445a1ed99d15cf/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:45.478157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeef2465-72f5-4e38-b2e4-078d5d1c84f2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.514347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.515033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.515319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.515624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.516340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:45.516658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.518019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fe456-8689-4f6e-8d1c-520c9e99c3b5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.531562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 414/838] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:45.540549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.645263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50472b59-0f2b-485a-8719-524b4126824e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.660794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.673428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004822-5722-4437-bbef-3f5015f98437 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.673706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004822-5722-4437-bbef-3f5015f98437 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.674015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a004822-5722-4437-bbef-3f5015f98437 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.676173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 415/839] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:45.683957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.684363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.684739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.686834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.687721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.688151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.688572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.697367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.699002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.704692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:45.706092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:45.712162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:45.713391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:45.714566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:45.716898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 425/840] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:45.722486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.766735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.803709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 416/841] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:45 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:45.820674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.850518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7232dea5-5601-44fd-af01-cbc628c53288 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.895838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.897342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.897833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.898312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.899805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:45.900343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.902643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.922880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05142f0b-0451-400e-b03c-9aa17fa0aaee None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:45.932634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 426/842] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:45.934812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:45.963511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:45.964224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:45.964507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:45.964783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:45.965418 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:45.966780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:45.976614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:45.977606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:45.978449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:45.980065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:45.982852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 417/843] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:45.984235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.033287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.081211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86303e58-95cd-4635-8d40-e0c35a94f782 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.109170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eda984-8dd7-4918-8180-c8239303184c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.119309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.120037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.120257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.120533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.147098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.148016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.148453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.148864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.149789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:46.150281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.151809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.157962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 418/844] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:46.164699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.197882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.199344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.230405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.274193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67145d7-56eb-4a60-8734-4ec486715982 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.281877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.302931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 427/845] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:45 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:09:46.313294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.317687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.318311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.318591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.318862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.319831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:46.320135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.321491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773f059c-5cc4-4122-bdbd-cfef93682e82 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.326287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 419/846] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:46.397025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed727ea-d55f-4849-bc43-898e254661ae None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.397311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed727ea-d55f-4849-bc43-898e254661ae None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.397599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed727ea-d55f-4849-bc43-898e254661ae None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.399267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 420/847] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:46.405067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.405362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.405645 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.424143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:46.425897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:46.427821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcf93-f9b2-4a96-a8ff-5c97a6b97631 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.475834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.480389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.481446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.482950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.482950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.497555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.499008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.501539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 421/848] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:46 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:46.520322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:46.521353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:46.522252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:46.522252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:46.522252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:46.522252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:46.522912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 428/849] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:46.527956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.529496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.643756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db91bf6c-bca6-45d7-b852-5b956406967a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.645662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b4efe2-f649-4cd0-919c-10fb6d181c2c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.647878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.680230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.681062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.681670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.681953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.682929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:46.683238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.685352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.689728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.691076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.691446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.691805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.693000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.695163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.702451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 422/850] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:46.708786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:46.710403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:46.711929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:46.713081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:46.717327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 429/851] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:46.718167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.777942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:46.849060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e040e77-aaf7-48d3-9f69-20b4d0e328ec None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.860171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcef898-3289-43a7-b574-a9587c5d2647 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:46.885993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.886616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.886889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.887158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.887854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:46.888153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.889506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.899787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 430/852] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:46.900789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.908112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:46.909006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:46.909434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:46.909855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:46.919878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:46.922129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:46.939729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:46.941342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:46.942273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:46.945385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 423/853] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:46.950563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:46.982409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:47.025639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3a29dd-232a-4ab3-a21b-e64636c2ed40 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:47.045235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:47.074155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:47.075022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.075882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.076312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.077494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:47.077953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:47.080290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:47.087448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 431/854] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:47.089765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879cea84-97e3-416a-a6c5-6478757735fd None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:47.135320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:47.136306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.136820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.137302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.138331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:47.138862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:47.141016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00a78-7cdc-490f-8ddd-f321288a603c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:47.161742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 424/855] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:47.167772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:47.279823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acff9bf7-3359-4594-b171-a2958ac954b5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:47.315979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:47.316597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.316883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.317159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.317954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=eb9910cbfb414432b3425c7dfdf79787) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:47.318264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:47.320042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec7e4c6-99d3-4623-a425-07fa19a54da6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:47.326467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 432/856] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:47 2022] GET /identity/v3/role_assignments?user.id=eb9910cbfb414432b3425c7dfdf79787&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:47.506708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfaec8-3115-4750-9027-cfb8fbfaad7d None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.507685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfaec8-3115-4750-9027-cfb8fbfaad7d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.508171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdfaec8-3115-4750-9027-cfb8fbfaad7d None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.510014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 425/857] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:47.515579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.516442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.516947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.536887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:47.538082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:47.574488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:47.615180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 433/858] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:47 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:47.633626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfaec8-3115-4750-9027-cfb8fbfaad7d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:47.722065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:47.743078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdfaec8-3115-4750-9027-cfb8fbfaad7d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:47.778184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:47.778759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.779040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.779321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.790920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:47.797260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:47.798174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:47.798997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 426/859] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:47.805376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:47.910977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94f302a-8e61-4ae6-bbe1-0443ad3a0cf4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:47.945799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:47.946416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:47.946690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:47.946957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:47.947693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:47.947990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:47.949343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:47.958516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 434/860] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:09:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:47.963958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.069704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839a5631-cfc5-4b3f-a20c-443f5d0b1505 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.105151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.105808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.106125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.106413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.115028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:48.117507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.118947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:48.134721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:48.135704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:48.136666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:48.136666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:48.136666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:48.136666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:48.136666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 427/861] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:48.148225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.192279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:48.253407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b439d-907e-4673-accb-a4b60e80ef2c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.292942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.293917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.294426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.294907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.295876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:48.296390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.297948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288a92ad-57bd-4bd9-81b8-51b17f1d1c38 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:48.313887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 435/862] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:48.320564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.425841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e62b36-5baf-411b-830c-ebeaa417fd3b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.460205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd9e0-6792-44f2-9d85-c5275e861322 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.460868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd9e0-6792-44f2-9d85-c5275e861322 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.461374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd9e0-6792-44f2-9d85-c5275e861322 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.461617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.462265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.462551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.462832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.467751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 436/863] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:48.469371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.469918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.470528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.471409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.472869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:48.484334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:48.485321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:48.488362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 428/864] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:09:48 2022] GET /identity/v3/users/swift => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:48.491830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.495268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:48.497462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:48.556516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:48.591577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 437/865] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:48 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:48.607404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.613719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ccd1d5-311e-4ea0-98a0-bbe87a2d679d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.632319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:48.652466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.653448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.653946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.654477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.655611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:48.656173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.657818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:48.677246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 429/866] 10.222.0.40 () {56 vars in 1135 bytes} [Sun Jul 31 14:09:48 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:48.685014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:48.723110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9733a27d-df3a-4878-b68a-6f2725e25e27 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.772871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.773555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.773844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.774121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.775113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.776846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:48.799698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 438/867] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:09:48 2022] POST /identity/v3/services => generated 251 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:09:48.818724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4bcc7a-4221-450f-a935-2d8803765361 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:48.832882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:48.866773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:48.867389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:48.867700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:48.867976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:48.945842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:48.947309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f061f7-6edd-44a1-93f0-0b2ce510038a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.056381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 430/868] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:48 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:09:49.062101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.161308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.181378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68f47b8-a58e-4477-a6ea-31bad994ed20 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.186383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa67b-fafd-405b-aef8-2e59b5f1852e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.186671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa67b-fafd-405b-aef8-2e59b5f1852e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.186949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340fa67b-fafd-405b-aef8-2e59b5f1852e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.188335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 431/869] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:49.196655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.196944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.197221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.212878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.219180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:49.220521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:49.233477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.234595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.235085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.235652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.248712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.250735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:49.267100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:49.268125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:49.269006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:49.269006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:49.269006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:49.269006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:49.271505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 439/870] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:49.277144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.303279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 432/871] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:49 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:49.325185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.409730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b2621b-25f7-480a-859b-1594ce4c98b2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.429222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45f4cd-35e6-4975-80ee-59970f9976b2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.452973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.453791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.454388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.454722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.456340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:49.456656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.458789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a34075a-3af5-4fda-8625-78168037fc86 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.460495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.466988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.467636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.467922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.468198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.468906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.470263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53060eb3-ccd0-4499-8e90-e91e0eb660bd None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.474869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 440/872] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:49.475860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.477175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947c47-e0fa-4e34-ae2d-0a7bd79bd265 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.477508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947c47-e0fa-4e34-ae2d-0a7bd79bd265 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.477816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93947c47-e0fa-4e34-ae2d-0a7bd79bd265 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.479578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 441/873] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:49.481190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93947c47-e0fa-4e34-ae2d-0a7bd79bd265 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.492137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 433/874] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:09:49 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:09:49.493755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.494269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.494759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.514366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:49.516124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:49.601557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 434/875] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:49 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:49.602175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93947c47-e0fa-4e34-ae2d-0a7bd79bd265 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.624513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.648032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.648727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.649012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.649291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.657966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.659919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:49.674089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:49.675096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:49.676047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:49.679670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 442/876] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:49.681036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.749608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.759177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7d1ea-6881-4d81-bfd1-f29d79757e8d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.787800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:49.796561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a1b007-be2c-486d-8842-8965ee62e6ad None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.798534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.799550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.800056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.800551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.812151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:49.818394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:49.819653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:49.820525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:49.823326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 435/877] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:49.834944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.847086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.848428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.848971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.849395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.850508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:49.851013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.853294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:49.870063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 443/878] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:49.878934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:49.948798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe720ac0-8d31-4fc7-a868-5427a085f234 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:49.989864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:49.991115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:49.991847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:49.992306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:49.993340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:49.993798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:49.995914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.000651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34dc0953-eed8-4f80-afd4-564af9681633 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.007396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 436/879] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:50.012836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:50.018960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:50.045443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.046340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.046706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.047058 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.048286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=9d498354314244afa4445a1ed99d15cf) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:50.048818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.051685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.067503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 444/880] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:49 2022] GET /identity/v3/role_assignments?user.id=9d498354314244afa4445a1ed99d15cf&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:50.076041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:50.154409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7af530a-3a08-46c4-a1f6-e7c7e8e04eab None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.196214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.196883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.197165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.197446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.210248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.211680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:50.227696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:50.228666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:50.229560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:50.229560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:50.229560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:50.229560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:50.229560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 437/881] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:50.234702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:50.346117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f7d53f-6904-43b7-8d10-9ad6b811794d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.381887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.382895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.383386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.383899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.384849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:50.385377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.386938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.399285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 445/882] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:50.404504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:50.504264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:50.509942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13dd5774-b294-4cd0-8ae1-581b6a848617 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.552777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.553499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.553772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.554031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.563536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.566061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:50.585510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:50.586511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:50.587494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:50.588734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 438/883] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:50.593142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:50.635515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:50.699689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7101d5dc-0dad-4f71-8499-8cfaf3c55e52 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.735128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.735777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.736054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.736321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.737679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:50.737979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.739400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23ea62c-e234-4a5f-9090-4893d0eca86a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:50.756866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 446/884] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:50.762406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:50.868225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbb2839-f80a-45f0-89bb-fba3384ac39e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:50.902779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:50.903406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:50.903715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:50.903992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:50.930993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:50.986599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:50.988314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.065159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eae2-5e32-40dd-b0b2-ba656335d84e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.065390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eae2-5e32-40dd-b0b2-ba656335d84e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.065732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4751eae2-5e32-40dd-b0b2-ba656335d84e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.067223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 447/885] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:51.072779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.073172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.073547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.090723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 439/886] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:50 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:09:51.091889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:51.093101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:51.096912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.155272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.175734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 448/887] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:51.194320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.202325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40310e3-a7ee-4ebc-a977-c89788ff72ca None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.223149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.241258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.241921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.242202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.242478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.254175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.256049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:51.273371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:51.274592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:51.275510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:51.275510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:51.275510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:51.275510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:51.275510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 440/888] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:51.284719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.314088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2be0e0-d0a4-4185-bca0-ecaa8e5dc50b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.353925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.354964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.355538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.356100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.356946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.358563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:51.368714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:51.369750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:51.370572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:51.378411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 449/889] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:51.380623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.397561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b34e13-4296-42dd-b9d3-6d9b8055edee None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.407120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.434668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.435454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.435743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.436023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.436723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.437263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.438614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.455847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 441/890] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:51.458679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.492848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.506025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82abd109-5e31-4e78-a40e-a036bdb82453 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.549418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.550528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.551004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.551512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.552507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.553026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.554677 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.565424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 450/891] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:51.566596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.575333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-821922a5-4d98-43eb-8abf-613cf2ae9cf6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.624067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.624720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.625095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.625549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.633847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.635281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7aaf7a2b-710b-470f-9958-e0ed76bfdc00 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:51.647389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:51.648351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:51.649258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:51.653281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 442/892] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:51.654437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc4e32-fb45-4295-9609-4ee43d0169fd None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.654717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc4e32-fb45-4295-9609-4ee43d0169fd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.654992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebc4e32-fb45-4295-9609-4ee43d0169fd None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.656277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 443/893] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:51.656964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc4e32-fb45-4295-9609-4ee43d0169fd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.681797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec677a62-6aec-4d67-bd5d-a5ef9a6ddeec None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.687689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.718366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.718982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.719263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.719573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.720249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.720553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.721945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4887f942-d80d-4036-9406-527583b3dcca None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.726657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 451/894] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:51.729243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.729243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.729243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.746349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:51.747573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:51.761110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.772717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebc4e32-fb45-4295-9609-4ee43d0169fd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.808782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.809450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.809736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.810008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.811164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.811335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.812765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.827668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 452/895] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:51 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:51.827668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.827933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 444/896] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:51.830993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.922984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:51.935775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87335aea-4bbf-4887-84c3-05994ba8c33b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.937556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28fb400-99ca-49e4-b2b5-67edc0767176 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:51.974483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.974792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:51.975440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.975705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:51.975815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.975992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.976345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:51.976784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] RBAC: Adding query filter params (role.id=8b7c08cb494340d58b4cb2b6d425c5c0, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=eb9910cbfb414432b3425c7dfdf79787) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.976948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:51.977097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.977998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] RBAC: Adding query filter params (interface=public, service_id=3a3699b2ff4f463f93b7e08cbfe5bf54, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:51.978453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:51.978575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.980165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:51.984325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 445/897] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/role_assignments?user.id=eb9910cbfb414432b3425c7dfdf79787&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=8b7c08cb494340d58b4cb2b6d425c5c0 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:51.985320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:51.995649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 453/898] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/endpoints?service_id=3a3699b2ff4f463f93b7e08cbfe5bf54&interface=public®ion_id=RegionOne => generated 194 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.001255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.018870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.122159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f553d1-7857-4c65-8cb5-14643d4998b5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.129707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c67c5d8-da49-494e-b630-00a166cdac1e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.162118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.163292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.163991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.164436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.165714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.167382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.170052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.170879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.171275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.171791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.172453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.174519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-5c244082-269c-4762-ad3c-f601bcc11a7c None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:52.179509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:52.180632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:52.181686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.191474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 454/899] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services => generated 1268 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:09:52.191714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 446/900] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 31 14:09:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:52.198581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.258127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008a8d-e344-4ae5-b9bd-a628ccc8ad97 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.258461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008a8d-e344-4ae5-b9bd-a628ccc8ad97 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.258679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e008a8d-e344-4ae5-b9bd-a628ccc8ad97 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.265718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 447/901] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:52.268009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.268291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.268564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.289009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.294118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:52.295788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:52.315117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a05a73e-3dc7-47fe-ba46-1ccb8e77783d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.319889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.367236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.367615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.367931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.368037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.368698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:52.368995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.370354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.379478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 455/902] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.382386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.395654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 448/903] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:52 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:52.414904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.507619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0770618-a285-43f5-995e-2c7cea88209f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.527613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fa0c06-98df-44f9-947c-910fc0c3ae59 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.547300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.548193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.548724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.549179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.550100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:52.550585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.552339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.563148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 456/904] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.564334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.568207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.569100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.569383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.569659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.570329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.570436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.571820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:52.581950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:52.582971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:52.583895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 449/905] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:52.588892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.611822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.699789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95334a7-1b61-4c89-8ab1-4c69f4191810 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.710136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e28bea-8b95-4f78-9792-a400e7fa7805 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.738842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170314-480c-4017-87dc-574606a1e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.739502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170314-480c-4017-87dc-574606a1e607 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.739835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170314-480c-4017-87dc-574606a1e607 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.740114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99170314-480c-4017-87dc-574606a1e607 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.740992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170314-480c-4017-87dc-574606a1e607 None admin] RBAC: Adding query filter params (interface=public, service_id=563517d444b547668f773620fb228044, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:52.741309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170314-480c-4017-87dc-574606a1e607 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.742683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99170314-480c-4017-87dc-574606a1e607 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.753459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 457/906] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/endpoints?service_id=563517d444b547668f773620fb228044&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.754556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170314-480c-4017-87dc-574606a1e607 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.757706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.759462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.759887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.760303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.761125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:52.761569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.763082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.775459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 450/907] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.777362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:52.848507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.859879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:52.888818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6c0cb-dc6a-40a2-9ffa-12a7d3da607b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.891172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99170314-480c-4017-87dc-574606a1e607 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:52.925507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.926365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.926796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.927216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.928105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:52.928562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.930749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead177a4-7bd7-4173-a7ee-d9547b512ce6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.935935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:52.936559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:52.936836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:52.937109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:52.937765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:52.939151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f10974f-2052-4d52-887b-a3aeb0c0c1e7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:52.940086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 451/908] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:52.955936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 458/909] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:09:52 2022] GET /identity/v3/services => generated 1268 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:09:54.008529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd2de-e7b4-4995-807f-2f0534aa476b None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.009128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd2de-e7b4-4995-807f-2f0534aa476b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.009128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cd2de-e7b4-4995-807f-2f0534aa476b None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.010642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 452/910] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:54.015715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748fbad-5a05-4030-85c3-0ed3f4b11c46 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.016103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748fbad-5a05-4030-85c3-0ed3f4b11c46 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.016266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6748fbad-5a05-4030-85c3-0ed3f4b11c46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.035687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6748fbad-5a05-4030-85c3-0ed3f4b11c46 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:54.036861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6748fbad-5a05-4030-85c3-0ed3f4b11c46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:54.108712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 459/911] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:54.130547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd2de-e7b4-4995-807f-2f0534aa476b None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.242141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cd2de-e7b4-4995-807f-2f0534aa476b None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.274597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a0f5a-fad6-40c2-a585-b32768a26369 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.274780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a0f5a-fad6-40c2-a585-b32768a26369 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.274982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8a0f5a-fad6-40c2-a585-b32768a26369 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.276610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 460/912] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:09:54.280197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.281058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.281267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.281554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.281834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.282124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.282584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.291541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.295043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:54.301257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:54.302189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:54.303031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:54.304273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:54.305451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:54.309741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 453/913] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:54.315680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.375005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.393205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 461/914] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:54 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:54.415608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.448107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f579b-86fa-4c80-9ecf-aa08492c3cdc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.492254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.493809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.494280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.495199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.496469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:54.497313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.499361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.511178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 454/915] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:54.517086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.524679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c74fbc8-512a-4124-b4ae-a0d10ba77209 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.569035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.569939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.570224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.570495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.571125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.573117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:54.585177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:54.586211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 31 14:09:54.587025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:54.591497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 462/916] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:54.593987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.601376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.659875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-294c02a2-2049-4a53-8100-6a5489a930dd None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.668753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.706227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.708313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.709115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.709538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.724580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.725866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe3e5f7-2795-44a3-b88e-e7306a92b4ab None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.726486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:54.746512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:54.747634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:54.748732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:54.748732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:54.748732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:54.748732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:54.755817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 455/917] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:54.756642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.774086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.774765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.774991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.775263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.775988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:54.776298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.777662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.782929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 463/918] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:54.788668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.895581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa1c220-7eca-497f-9924-e0dbb4b2e107 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.897252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664393f5-7476-4526-801a-ca68917ed176 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:54.912753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.919663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:54.941258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.942087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.942422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.942795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.943634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:54.944133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:54.944376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.945063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:54.945490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:54.945909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:54.946112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.946883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:54.947328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:54.949738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:54.957101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 464/919] 10.222.0.40 () {56 vars in 1114 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:54.961867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 456/920] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:54.962838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:54.968526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.072858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef83416-a8b7-4364-9354-b91be4d58aca None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.085218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f606e034-97a7-4f9f-8f30-c0f91008292d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.106520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.107160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.107456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.107747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.115200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.116661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.118346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.118961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.119242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.119555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.120265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.124495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7f9b62f9-f73d-4ad1-866e-139778eaff8b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:55.128941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:55.129882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:55.130827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 457/921] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:55.136025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3965-610e-41f0-93bb-99c96cf81fa3 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.136025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3965-610e-41f0-93bb-99c96cf81fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.136025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fab3965-610e-41f0-93bb-99c96cf81fa3 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.139234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 458/922] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:55.140524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6eaf-70c3-4810-b827-c6ea2e355e82 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.140807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6eaf-70c3-4810-b827-c6ea2e355e82 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.141082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd6eaf-70c3-4810-b827-c6ea2e355e82 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.142387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 459/923] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:55.143439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ee15-8a04-4a1b-b952-eda6f85427fe None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.143770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ee15-8a04-4a1b-b952-eda6f85427fe None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.144042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295ee15-8a04-4a1b-b952-eda6f85427fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.152733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 465/924] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:09:54 2022] POST /identity/v3/endpoints => generated 346 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:55.153807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.163826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0295ee15-8a04-4a1b-b952-eda6f85427fe None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:55.165047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0295ee15-8a04-4a1b-b952-eda6f85427fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:55.194490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.211328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.246881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 460/925] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:55.248380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.248663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.248936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.271318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:55.272562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:55.289351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cebd8e-24e2-49ee-9218-afcbe7a6b245 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.329358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.329907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.330961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.331438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.332279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:55.332642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.334033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.353845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 466/926] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:55.354594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.359202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 461/927] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:55 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:55.360363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.452915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.462681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.466650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d9de072-f8c9-437e-bb30-cc372bb66346 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.468025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e199a00f-6c1e-4c36-a9ba-d6a6d838eef3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.503180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.503842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.504138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.504388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.505081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.506447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.509505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.510148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.510424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.510702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.521623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.531813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 467/928] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:09:55 2022] POST /identity/v3/services => generated 241 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:55.532020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.532117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:55.550488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:55.551520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:55.552439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:55.552439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:55.552439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:55.552439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:55.556999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 462/929] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:55.557930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.685974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee87f30-f3bb-4a15-bc1d-6a7c89d4ed69 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.688476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0247c805-9479-4761-8712-2ee99b6ce68d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.730855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.731067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.731295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.732176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.732336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:55.732639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.733990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.737098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.737756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.738037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.738313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.738945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.740531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.744492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.748772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 463/930] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:55.753847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:55.754801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 31 14:09:55.756090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:55.756991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.757063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 468/931] 10.222.0.40 () {56 vars in 1132 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:55.763327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.785373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:55.879650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e553d2-fc1b-4853-88f1-264770df8acd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.887625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b002e2b4-de57-476b-9699-48eafb885521 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:55.918533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.919149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.919436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.919732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.920407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:55.920707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.922052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.931714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 469/932] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:55.932562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:55.932716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:55.933491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:55.933925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:55.934342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:55.944849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:55.947240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:55.961496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:55.962621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:55.963620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:55.967533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 464/933] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:55.968413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:56.029805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:56.043091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:56.064894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003feb11-f039-43a6-97da-c9b1f7dbc15e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:56.070504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246b33f0-00f4-4dfd-9420-515ef0b76444 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:56.103239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:56.104444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:56.104801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:56.105333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:56.105446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:56.105731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:56.106008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:56.106124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:56.106799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:56.107090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:56.107192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:56.107651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:56.108549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:56.109299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:56.115273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 470/934] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:56.120411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:56.136455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 465/935] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:09:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:56.137429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:56.246517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f553858b-d42e-4f49-a339-25c9f3ff8e21 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:56.249295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b09f2e6-468e-4e78-aa05-296b61788033 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:56.267781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:56.283980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:56.284751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:56.285030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:56.285299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/eb9910cbfb414432b3425c7dfdf79787/roles/8b7c08cb494340d58b4cb2b6d425c5c0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:56.287643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=eb9910cbfb414432b3425c7dfdf79787, role_id=8b7c08cb494340d58b4cb2b6d425c5c0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:56.289283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309ad3ef-e8f4-4513-9cce-58e2f1de7b80 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:56.294880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:56.295626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:56.295973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:56.296283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:56.296923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:56.298428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea34310-443f-4c4c-bd49-887617829aff None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:56.307240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 466/936] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:09:56 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/eb9910cbfb414432b3425c7dfdf79787/roles/8b7c08cb494340d58b4cb2b6d425c5c0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:09:56.331761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 471/937] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:09:56 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:09:57.245363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573807f8-b39c-4156-a7c7-54e72a49b5ef None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.245941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573807f8-b39c-4156-a7c7-54e72a49b5ef None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.245941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573807f8-b39c-4156-a7c7-54e72a49b5ef None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.247539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 467/938] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:57.253192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.253608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.253734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.260350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:57.274501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:57.275712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:57.350528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 472/939] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:57.373870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573807f8-b39c-4156-a7c7-54e72a49b5ef None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:57.478919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573807f8-b39c-4156-a7c7-54e72a49b5ef None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:57.516861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:57.518085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.518370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.518648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.519345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:57.520848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:57.523709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING keystone.server.flask.common [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 31 14:09:57.543017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.sql.core [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 14:09:57.543017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 14:09:57.543017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [parameters: {'id': '0b46e955243943bfbba8c998a23d151f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 14:09:57.543017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80121) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-32bc554c-5dce-4bda-8f04-aac4a4fb421f None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b46e955243943bfbba8c998a23d151f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b46e955243943bfbba8c998a23d151f. Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:09:57.560525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application result.read() Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 31 14:09:57.561394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application yield resource Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise self.value Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application session.commit() Jul 31 14:09:57.562359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self.session.flush() Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application compat.raise_( Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 14:09:57.563559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 14:09:57.564978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application result.read() Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:09:57.565758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [parameters: {'id': '0b46e955243943bfbba8c998a23d151f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.566655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:57.572276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0b46e955243943bfbba8c998a23d151f. Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 468/940] 10.222.0.40 () {60 vars in 1140 bytes} [Sun Jul 31 14:09:57 2022] POST /identity/v3/projects => generated 262 bytes in 189 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:57.574155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:57.660116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b47e-2a32-46c5-b277-8f31523ef39d None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.660404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b47e-2a32-46c5-b277-8f31523ef39d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.660657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da8b47e-2a32-46c5-b277-8f31523ef39d None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.665202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 469/941] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:57 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:57.670642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.671077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.671363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.675235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86626594-8109-4806-aa1d-ec78822624e5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:57.695984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:57.698207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:57.703474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:57.712663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:57.713307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.713591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.713866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.721067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:57.722556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:57.733074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:09:57.733729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:57.735551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:57.738064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 473/942] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:57.742016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:57.787519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 470/943] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:57.809084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:57.871749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d1af64-0a18-4f79-b9c7-45fc8480f35c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:57.915752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:57.917123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.917527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.917810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.918807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:57.919120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:57.921314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:57.926060 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78133fb4-42f0-4339-b78a-baa0c2439b6f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:57.947520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 474/944] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:57 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:57.968708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:57.969382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:57.969666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:57.969941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:57.971193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:57.972797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:57.983480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:57.985816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:57.987875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 471/945] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:09:57 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:57.991766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.049732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.106324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0852aeed-366c-401a-a0b8-8ba7ddb5c231 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.148944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.148944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.148944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.148944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.149800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:58.149905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:58.151215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:58.156654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 475/946] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:09:57 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:58.163793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.199611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.276920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085030e7-a652-4736-b03a-bc5a89f749a2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.312761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.313436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.313717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.313989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.314775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:58.315078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:58.316463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13771c1-bafa-4ebd-8927-950aab9c1d36 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:58.322355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 472/947] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:58.328573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.437144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a08092-87d3-45be-8c54-22b431f38e9b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.442708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb4a2-cfb3-45a9-9fae-75526572e094 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.442998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb4a2-cfb3-45a9-9fae-75526572e094 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.443278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdcb4a2-cfb3-45a9-9fae-75526572e094 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.445263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 473/948] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:58.450219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f10a9-3170-4b04-81f5-a3690c1aa956 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.450503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f10a9-3170-4b04-81f5-a3690c1aa956 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.450781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863f10a9-3170-4b04-81f5-a3690c1aa956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.454640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.471971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-863f10a9-3170-4b04-81f5-a3690c1aa956 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:58.473383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-863f10a9-3170-4b04-81f5-a3690c1aa956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:58.478128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.478754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.479036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.479313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.480118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] RBAC: Adding query filter params (interface=public, service_id=54b5839af3334077b8df30b6d6356e8e, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:58.480430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:58.481790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b084ef-668c-47d9-bfe3-b6d29d39c181 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:58.492264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 476/949] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/endpoints?service_id=54b5839af3334077b8df30b6d6356e8e&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:58.492264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bfb33-7551-44b6-8220-de9ffa808a0e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.492264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bfb33-7551-44b6-8220-de9ffa808a0e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.492264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bfb33-7551-44b6-8220-de9ffa808a0e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.492264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 477/950] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:09:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:09:58.493084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bfb33-7551-44b6-8220-de9ffa808a0e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.559662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 474/951] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:09:58.560894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.561187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.561465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.588622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:09:58.589715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:09:58.608920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.610470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bfb33-7551-44b6-8220-de9ffa808a0e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.646966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.647614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.647893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.648166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.648938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:58.650296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:58.667745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 478/952] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/services => generated 1497 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:09:58.667745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.679609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 475/953] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:09:58 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:09:58.697796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.726612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.804054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b7c276-77be-4ddd-ac35-29c768421e36 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.840763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73350e7b-ed46-4a59-8627-429cf445825a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:58.846974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.847655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.847934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.848204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.859313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:09:58.864247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:58.865247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:58.866084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 479/954] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:58.871750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.890775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:58.891622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:58.892348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:58.893207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] PATH_INFO: `/v3/services/compute` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:58.893950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:58.896203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:58.908965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:58.910255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:09:58.911188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:58.911971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 476/955] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/services/compute => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:58.917185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:58.946943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.991217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:58.994642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fe78bd-3e3d-456d-b956-ca92453bd459 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.031645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.032264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.032543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.032813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.034066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:59.034365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.035967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.046269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 480/956] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:59.052872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.061998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b71241a-f1c3-49f4-9f68-e0e66f0f2fba None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.098170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.098648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.099042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.099405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.100205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] RBAC: Adding query filter params (name=compute) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:59.100602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.102184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.112121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 477/957] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:58 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:09:59.114734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.191860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a51ae6-3d63-44ee-8f21-ba414e4dd716 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.217301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:59.229529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.230252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.230445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.230745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.238928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2671e58-ccf9-4a91-8976-5d853e31c6cc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.242189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.243796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:09:59.258536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:59.259699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:59.260718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:09:59.260718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:09:59.260718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:09:59.260718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:09:59.260718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 481/958] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:09:59.265452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.278404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.279667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.280149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.280619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.281521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:59.282032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.284305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.288398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:59.289762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 478/959] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:59.387465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8b2963-2cb3-421a-8b8f-e90a311d68e0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.430921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.431597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.431882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.432164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.432875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:59.433181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.434585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.453766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 482/960] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:59.456566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.563224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c451-99e9-4f72-b319-0f6e1cfd08a3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.599219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.599892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.600176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.600454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.612344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.614054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:09:59.625688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:09:59.627169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 31 14:09:59.628190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:09:59.630746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 479/961] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:09:59.636481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.727802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:59.753978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ed158-6c13-4d19-b299-166244d2457b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.789836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.790456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.790760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.791061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:09:59.791956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:09:59.792267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:09:59.793771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33edbef9-b271-413e-9555-5667bf339f5d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:09:59.815936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 483/962] 10.222.0.40 () {56 vars in 1149 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:09:59.819574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:09:59.832126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:09:59.931225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720f10ab-dc14-4d5e-80fd-4ced128acff7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:09:59.966699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:09:59.967734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:09:59.968250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:09:59.968752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.038167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.039982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.053024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.082752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1dd51-b10d-449d-b462-c81b460c0fea None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.083080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1dd51-b10d-449d-b462-c81b460c0fea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.083314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a1dd51-b10d-449d-b462-c81b460c0fea None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.084939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 484/963] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:00.089624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.089880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.090159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.109232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:00.110419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:00.159766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 480/964] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:09:59 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:00.162255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.211900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 485/965] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:00.236058 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.272681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e3af51-1cf8-4b44-8c2d-3e067fdee507 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.281192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.313074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.313980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.314324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.314696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.320694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.326822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.329437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:00.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:00.348166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:00.349297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:00.349297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:00.349297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:00.349297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:00.350852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 481/966] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:00.359539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.368565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571a501a-5f47-4f1c-9c7d-90987612ee7e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.405118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.405887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.406282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.406656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.407460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.409675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.412378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING keystone.server.flask.common [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 31 14:10:00.426025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.sql.core [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 14:10:00.426025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 14:10:00.426025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [parameters: {'user_id': 'beafd708a465434eb0866331f076d5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 14:10:00.426025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80120) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jul 31 14:10:00.433492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:10:00.434389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application yield resource Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application next(self.gen) Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.rollback() Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.force_reraise() Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise self.value Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application session.commit() Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 31 14:10:00.435367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._prepare_impl() Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.session.flush() Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._flush(objects) Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application compat.raise_( Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application flush_context.execute() Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rec.execute(self) Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 31 14:10:00.436286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 31 14:10:00.437244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result = self._query(query) Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application conn.query(q) Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application result.read() Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 31 14:10:00.438075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [parameters: {'user_id': 'beafd708a465434eb0866331f076d5da', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:00.438905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 31 14:10:00.439948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 31 14:10:00.441096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.442126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 486/967] 10.222.0.40 () {60 vars in 1131 bytes} [Sun Jul 31 14:10:00 2022] POST /identity/v3/users => generated 160 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:00.443887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.486919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb04587-89be-49ed-989f-e16409dd16f6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.522399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.523551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.524004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.524451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.525321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:00.525923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.527587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.546642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 482/968] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:00.548151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.553450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e9eafc-3093-4fe5-8846-083d4a28256a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.578549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.580380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.592361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.593894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.594182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.594461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.603166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.605084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:10:00.617741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:00.618717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 31 14:10:00.619703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:00.623653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 487/969] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:00.624680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.661073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2f49-c94c-4528-955d-2d3fdef916f2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.696377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.697371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.697868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.698334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.705527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.707191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:10:00.718625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:00.719597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:00.720513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:00.722601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 483/970] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:00.729090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.739300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ab7e1-03ee-434a-be20-413f8b69cbc7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.778336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.779291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.779812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.780391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.781357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:00.782024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.783714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae712e4d-711e-42b9-aaeb-8cab416e3cd7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.817687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.835485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 488/971] 10.222.0.40 () {56 vars in 1101 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:00.836824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7524-a5d5-47d8-b61e-4c92ecf86bfd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.837112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7524-a5d5-47d8-b61e-4c92ecf86bfd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.837391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f7524-a5d5-47d8-b61e-4c92ecf86bfd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.843281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 489/972] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:00.844956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.845248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.845530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.854478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148f97a2-9165-480a-a8fd-cb052578b50e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:00.866172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:00.867889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:00.898983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:00.900283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:00.900736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:00.901161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:00.902247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:00.902559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:00.904189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:00.919830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:00.924270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 484/973] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:00.925513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:00.975139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 490/974] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:00 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:00.995084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:01.043905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db8221f-9318-4079-895d-3d0fd623ea29 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.087046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.087623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.087953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.088183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.088981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] RBAC: Adding query filter params (role.id=8b7c08cb494340d58b4cb2b6d425c5c0, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=eb9910cbfb414432b3425c7dfdf79787) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:01.089292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.090731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.097876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 485/975] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/role_assignments?user.id=eb9910cbfb414432b3425c7dfdf79787&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=8b7c08cb494340d58b4cb2b6d425c5c0 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:01.128925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2414f24-dd56-48df-b8a9-361ceadbd278 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.171830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.172494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.172773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.173043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.174259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.175656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:01.185400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:01.186476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:01.187361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 491/976] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:10:00 2022] GET /identity/v3/services/object-store => generated 93 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:01.193008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:01.196315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:01.302840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d64bd-e134-4792-a296-88858c98d603 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.336668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.337356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.337586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.337894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.338577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:01.338879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.340360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-685d76ff-c10a-4fc3-9bf8-e4fda43bec40 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.346153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 486/977] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:01 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:01.351733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:01.386301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:01.452088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145f4c56-d80b-40f6-b1cc-57a720080160 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.484732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59b60-c53c-4b1e-8499-342ccb7d2525 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.485070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59b60-c53c-4b1e-8499-342ccb7d2525 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.485296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc59b60-c53c-4b1e-8499-342ccb7d2525 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.485917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.486537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.486817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.486979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 487/978] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:01.487140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.487858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:01.488279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.489640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.491939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.492230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.492513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.495300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 492/979] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:01 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:01.500573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:01.510728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:01.511951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:01.575414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:01.584446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 488/980] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:01 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:01.607084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:01.607611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:01.611181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be6582c-8fc8-47aa-ad99-67b9ad6f09cc None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.646526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.647151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.647553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.647732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.648446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.649797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89293184-1bb3-48a5-be9f-e840b1509b13 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.671463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 493/981] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:10:01 2022] POST /identity/v3/endpoints => generated 349 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:01.717407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e57fb2-6b47-46bf-87a5-11df778bb50e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:01.761308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:01.761939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:01.762217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:01.762487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:01.763184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:01.764625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb60f7-1029-4c5a-ba0b-a8d848e91ae9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:01.782557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 489/982] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:10:01 2022] POST /identity/v3/services => generated 242 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:03.147949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab22b-142d-4b79-aa3d-8bd462d537ad None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.148342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab22b-142d-4b79-aa3d-8bd462d537ad None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.148541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aab22b-142d-4b79-aa3d-8bd462d537ad None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.150006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 494/983] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:03.154974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.155353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.155567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.173869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:03.175046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:03.222287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:03.252547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 490/984] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:03.274916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab22b-142d-4b79-aa3d-8bd462d537ad None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:03.373488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aab22b-142d-4b79-aa3d-8bd462d537ad None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:03.407280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:03.407954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.408238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.408511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] PATH_INFO: `/v3/services/image` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.409140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:03.410494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:03.419248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:03.420483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:03.421335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:03.423505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 495/985] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 31 14:10:03 2022] GET /identity/v3/services/image => generated 86 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:03.426739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:03.525794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30a407-4243-4640-b689-9a6eeda64e0e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:03.558606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:03.559226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.559550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.559829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.560507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] RBAC: Adding query filter params (name=image) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:03.560809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:03.562164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadf3760-743f-41dd-857b-69ec85dcd086 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:03.571496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 491/986] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:03 2022] GET /identity/v3/services?name=image => generated 122 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:03.573878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:03.597041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:03.678687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acd968e-c015-4291-8a61-a74d30c3c136 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:03.698310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafafe6-1177-4d29-8125-81770e9b31bb None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.698624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafafe6-1177-4d29-8125-81770e9b31bb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.698872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baafafe6-1177-4d29-8125-81770e9b31bb None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.700385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 492/987] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:03.705073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.705359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.705633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.713936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:03.714559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.714841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.715114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.715819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] RBAC: Adding query filter params (type=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:03.716125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:03.717473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26299a7e-05af-43fa-99ce-9b2833059798 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:03.722609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 496/988] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:03 2022] GET /identity/v3/services?type=image => generated 122 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:03.732804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:03.734044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:03.774924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:03.817565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 493/989] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:03.836096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcd9c-790b-4526-a9d8-2ac9b93328f8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.836410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcd9c-790b-4526-a9d8-2ac9b93328f8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.836659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9bcd9c-790b-4526-a9d8-2ac9b93328f8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.838201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 497/990] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:03.838904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:03.843463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.843792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.844063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.862900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:03.864176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:03.937692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 498/991] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:03 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:03.945632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0629bda-6708-49e5-8ada-b5420ad54657 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:03.960253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:03.975668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:03.984809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:03.985467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:03.985802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:03.986573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:03.987207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:03.988625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:03.997712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:03.998826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:03.999715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 494/992] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:10:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:04.005154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.046600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.083286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5946f291-eb0b-4682-a522-eabea54fefe9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.116769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dff8b26-220a-428d-99f0-5c82f427b41f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.121123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.121773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.122055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.122383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] PATH_INFO: `/v3/services/compute` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.123624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.124570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:04.136041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:04.137278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:04.138344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:04.142261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 499/993] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:04.143579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.166215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.167270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.167585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.168007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.168808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.169119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.170965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.176628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 495/994] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:04.182014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.232844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.258045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefc13b3-a1e9-4e77-a2fc-4f25d54dcfac None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.291755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.294761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.295392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.295721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.295998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.296682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] RBAC: Adding query filter params (name=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.296985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.297453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2601b60-2279-40b2-8f1c-3b9379028769 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.298885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.303909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 500/995] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:04.309512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.337252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.337868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.338147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.338416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.339087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.339388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.340764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.346858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 496/996] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:04.352716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.425582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a3b7dc-3195-40fb-997f-ab4acc12feed None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.456034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fee295-81a1-4944-b42d-c913e4da1ff5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.462090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.462702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.463386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.463696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.464379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] RBAC: Adding query filter params (type=compute) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.465023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.466709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.477211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 501/997] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:04.478460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.488339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.494956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.495608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.496017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.496192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.496923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] RBAC: Adding query filter params (interface=admin, service_id=54b5839af3334077b8df30b6d6356e8e, region_id=RegionOne) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.497229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.498615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.503865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 497/998] 10.222.0.40 () {56 vars in 1249 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/endpoints?service_id=54b5839af3334077b8df30b6d6356e8e&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:04.509740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.540135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.597441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ff6679-28bd-4cb1-9c67-63ba23a97dcd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.623065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488d72fd-5e8f-4392-ba75-9dc9f2db1fa0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.632322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.632931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.633208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.633476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.634218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] RBAC: Adding query filter params (interface=public, service_id=70d0f0068c384581ab8acd71da04fd0e, region_id=RegionOne) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:04.634523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.635917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.641555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 502/999] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/endpoints?service_id=70d0f0068c384581ab8acd71da04fd0e&interface=public®ion_id=RegionOne => generated 194 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:04.647150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:04.660558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.661932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.662208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.662378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.663028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.664434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891a8366-6ad3-49ef-9fc8-2514cc931cd2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.671041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 498/1000] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services => generated 1727 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:10:04.732817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:04.769251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a65885f-9257-4f40-b413-cdc7f1912a89 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:04.809371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:04.810011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:04.810296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:04.810576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:04.811239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:04.812664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aa40479-c3f9-44a8-aaa6-563ff3cede50 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:04.831446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 503/1001] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:10:04 2022] GET /identity/v3/services => generated 1727 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:10:05.835888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7776-010b-47b2-b572-0728d58fe040 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:05.836420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7776-010b-47b2-b572-0728d58fe040 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:05.836420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26be7776-010b-47b2-b572-0728d58fe040 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:05.837836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 499/1002] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:05.845897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd77f9-f15a-4a2d-9b21-d7c8cf010bae None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:05.846426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd77f9-f15a-4a2d-9b21-d7c8cf010bae None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:05.846426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dfd77f9-f15a-4a2d-9b21-d7c8cf010bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:05.864338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0dfd77f9-f15a-4a2d-9b21-d7c8cf010bae None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:05.865521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0dfd77f9-f15a-4a2d-9b21-d7c8cf010bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:05.933252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 504/1003] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:05 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:05.956459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be7776-010b-47b2-b572-0728d58fe040 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:06.050979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:06.061694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26be7776-010b-47b2-b572-0728d58fe040 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:06.099376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:06.099992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:06.100257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:06.100528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:06.101236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:06.102590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc427ad3-131c-436d-ae10-72067d80993e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:06.127451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 500/1004] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:10:05 2022] POST /identity/v3/services => generated 242 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:06.748243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238760a7-01aa-4f8d-b594-097d23aec6d4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:06.748580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238760a7-01aa-4f8d-b594-097d23aec6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:06.748799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238760a7-01aa-4f8d-b594-097d23aec6d4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:06.750298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 505/1005] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:06.755938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc839f-5fb3-4ea9-adcd-0b0fa4f3a9a8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:06.756226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc839f-5fb3-4ea9-adcd-0b0fa4f3a9a8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:06.756501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fc839f-5fb3-4ea9-adcd-0b0fa4f3a9a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:06.774143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-85fc839f-5fb3-4ea9-adcd-0b0fa4f3a9a8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:06.775311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-85fc839f-5fb3-4ea9-adcd-0b0fa4f3a9a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:06.848190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 501/1006] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:06 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:06.870294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238760a7-01aa-4f8d-b594-097d23aec6d4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:06.879258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304ecfe-d031-4e69-b3f7-fd8a166ae206 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:06.879971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304ecfe-d031-4e69-b3f7-fd8a166ae206 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:06.879971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d304ecfe-d031-4e69-b3f7-fd8a166ae206 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:06.881351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 502/1007] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:06.886036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:06.886322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:06.886599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:06.904386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:06.905618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:06.939348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:06.974606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 503/1008] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:06 2022] POST /identity/v3/auth/tokens => generated 1656 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:06.980780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-238760a7-01aa-4f8d-b594-097d23aec6d4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:06.997205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.008642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:07.022672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.023321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.023638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.023910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.024542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.025982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:07.038111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:07.039138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 31 14:10:07.040024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:07.044851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 506/1009] 10.222.0.40 () {56 vars in 1126 bytes} [Sun Jul 31 14:10:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:07.045716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.133555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ade8ec-1a30-4c57-885a-d738f41f1272 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.150487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e566665-72ae-4489-9f02-09590e7a17c6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.171835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.172726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.173807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.174248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.175057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.176742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.177372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:07.186119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.186718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.186999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.187273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:07.187790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:07.188956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 31 14:10:07.189988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:07.190856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:07.190856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.190856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.192649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 504/1010] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:06 2022] GET /identity/v3/services/compute => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:07.198623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.200716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 507/1011] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:07 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:07.205639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.281286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:07.313066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40a8c3e-f050-48cb-8b52-b014f2b78953 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.317852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d8dae76-37a9-4f99-b351-1fd360be13a3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.351826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.352444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.352724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.352982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.353656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:07.353956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.356080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.361595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.362213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.362490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.362762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.363451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:07.363759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.365105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.370799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 505/1012] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:10:07 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:07.371364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.381150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 508/1013] 10.222.0.40 () {56 vars in 1122 bytes} [Sun Jul 31 14:10:07 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:07.382680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.441902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:07.485953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49782cc-07e1-4c7d-9583-78f8f5a9c024 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.489055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a88b2b-5e26-4052-8282-60308d7c0520 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.520664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.521288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.521565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.521836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.522519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:07.522830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.523677 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.524224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.524326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.524559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.524830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.525532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.526691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:07.526884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ac9b7-e965-45fd-94d8-2b1143799e14 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.530335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 506/1014] 10.222.0.40 () {56 vars in 1112 bytes} [Sun Jul 31 14:10:07 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:07.536587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:07.551455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 509/1015] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:10:07 2022] POST /identity/v3/endpoints => generated 325 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:07.652771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae62d188-6292-4ef7-a92f-e23003ba49f1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:07.695327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:07.695983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:07.696262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:07.696534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:07.697249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:07.698601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b28cf2-7c12-455e-ba00-237dc160e71f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:07.721436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 507/1016] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:10:07 2022] POST /identity/v3/endpoints => generated 334 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:08.183180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0956-80dd-4890-a1f8-f4544171b90a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.183748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0956-80dd-4890-a1f8-f4544171b90a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.183851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ae0956-80dd-4890-a1f8-f4544171b90a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.185362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 510/1017] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:08.191096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.191547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.191715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.209719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:08.210901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:08.258215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:08.286406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 508/1018] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:08 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:08.309142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae0956-80dd-4890-a1f8-f4544171b90a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:08.410511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ae0956-80dd-4890-a1f8-f4544171b90a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:08.428509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:08.457794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:08.458482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.458758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.459029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] PATH_INFO: `/v3/services/image` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.459679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:08.461042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:08.470480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:08.472066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:08.473133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 511/1019] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 31 14:10:08 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:08.480338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:08.577798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb30142-fa6d-413e-aae2-94b7406f4eec None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:08.611429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:08.612102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.612379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.612650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.613374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] RBAC: Adding query filter params (name=image) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:08.613733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:08.615135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:08.623552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 509/1020] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:08.626141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:08.734684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc677a6-942e-4344-a246-e46d60de0ede None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:08.770069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:08.770689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.770968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.771240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.771946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] RBAC: Adding query filter params (type=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:08.772251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:08.773608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:08.778376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 512/1021] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:08 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:08.785136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:08.831136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:08.888648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddba563-e621-455c-b45b-0e98d91a5dc5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:08.922432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:08.923080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:08.923362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:08.923666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:08.924428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] RBAC: Adding query filter params (interface=public, service_id=cce9e45b16f6427880eb5e89a7a97db3, region_id=RegionOne) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:08.924738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:08.926126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f955b6-50f3-4bd2-83ef-dda0d15e2040 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:08.931313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 510/1022] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:10:08 2022] GET /identity/v3/endpoints?service_id=cce9e45b16f6427880eb5e89a7a97db3&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:08.936949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:08.985874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:09.040392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f8bdb5-bbb9-4dab-89f0-d843928ede54 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:09.073305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:09.073923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.074201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.074474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.075123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:09.076511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9fc508-946d-499d-a729-3f7da4d3e06c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:09.087639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 513/1023] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:10:08 2022] GET /identity/v3/services => generated 1957 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:10:09.582329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd435-9b6b-4bc4-8a51-6cba3a2ed573 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.582852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd435-9b6b-4bc4-8a51-6cba3a2ed573 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.582852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7cd435-9b6b-4bc4-8a51-6cba3a2ed573 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.584380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 511/1024] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:09.589266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.589643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.589802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.607370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:09.608571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:09.676127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 514/1025] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:09.699227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd435-9b6b-4bc4-8a51-6cba3a2ed573 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:09.800622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7cd435-9b6b-4bc4-8a51-6cba3a2ed573 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:09.807801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:09.836441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:09.837001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.837276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.838679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:09.840119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c564414-97e7-4d8c-b755-5d88dedf1d2c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:09.841848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 512/1026] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:09.847546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:09.859962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334faafc-6bbd-4637-89af-89ee76abf77b None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.861002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334faafc-6bbd-4637-89af-89ee76abf77b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.861271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334faafc-6bbd-4637-89af-89ee76abf77b None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.862696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 513/1027] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:09.867239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.867559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.867837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.886372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:09.887571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:09.891461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:09.953853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 514/1028] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:09.954961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10394568-f8d6-48ff-9105-53d851a4e803 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:09.976865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:09.991245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:09.992130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:09.992411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:09.992685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:09.993371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:09.994808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.020887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 515/1029] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:10:09 2022] POST /identity/v3/projects => generated 311 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:10.084944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca8f0ec-4005-44e5-b21d-958330405860 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.088081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:10.130514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.131690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.132177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.132648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.149231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:10.154284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:10.155284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:10.156387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 515/1030] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:10:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:10.162106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:10.262122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:10.272482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12348aa-8c47-4719-b2a9-31d9858a15a0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.307526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.308141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.308419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.308687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.309394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:10.309747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:10.311100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.320035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 516/1031] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:10.325445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:10.427228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f048ad1-a403-4365-9791-97549643bafc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.461073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.461709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.461989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.462259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.472073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:10.473500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:10.488725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:10.489652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:10.490576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:10.490576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:10.490576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:10.490576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:10.490576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 516/1032] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:10.496741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:10.598891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aac022c-2614-47c8-805a-de57a76ec30f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.633307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.633984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.634263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.634535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.635268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:10.635601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:10.636966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.652679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 517/1033] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:10.653871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:10.674284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:10.759576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc855081-9b8f-4f71-a8ca-fa1ea6f4943d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.795393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.796376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.796376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.796645 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] PATH_INFO: `/v3/users/nova` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.805603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:10.807061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:10:10.818142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:10.819372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:10.821281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 517/1034] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:10.825489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:10.842039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:10.930144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa0c0fb3-1d58-4df3-8964-805f974646c1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:10.963219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:10.963928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:10.964210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:10.964481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:10.965275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:10.965578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:10.967056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ee05fc-32c9-4447-b447-929a32979ea3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:10.989663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 518/1035] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:10.990931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.069936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.095512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292ee39-eb9a-4a85-b39c-d72f1cb8d57e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.108697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8679d-3525-4fdd-86ad-fda729b5215e None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.108977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8679d-3525-4fdd-86ad-fda729b5215e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.109250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e8679d-3525-4fdd-86ad-fda729b5215e None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.110739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 519/1036] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:11.115590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.115877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.116150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.134865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:11.136246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:11.137303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.137934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.138476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.138754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.209225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.215467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 520/1037] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:11 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:11.224906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.226377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.238985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.311255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.335624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 518/1038] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:10 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:10:11.341076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.353335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d5fdc6a-febd-4dba-9392-54c845eb6c97 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.387146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.387821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.388102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.388370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] PATH_INFO: `/v3/services/image` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.389003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.390362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:11.399628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:11.403620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:11.404671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 521/1039] 10.222.0.40 () {56 vars in 1105 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/services/image => generated 86 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:11.406959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.446448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a49e550-0c7f-4699-866b-45f550b131b1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.472963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.484225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.484877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.485157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.485498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.498689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.500216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.513114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce010a0-4479-4c94-a7e1-730419bd54bd None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:11.515285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:11.516457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:11.518005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:11.518005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:11.518005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:11.518005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:11.518005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 519/1040] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:11.522611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.551796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.554156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.555116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.555461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.555791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.556470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] RBAC: Adding query filter params (name=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:11.556782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.558142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.568866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 522/1041] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:11.569857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.633800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c8d73e-5ad1-4469-9ee7-b4175cca8de6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.670481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44658401-f74a-42b3-8e14-fc6e5245f8da None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.671004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.671549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.671941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.672216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.673070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:11.673374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.674740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.693048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 520/1042] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:11.694157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.699977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.709705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.710325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.710609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.710895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.711595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] RBAC: Adding query filter params (type=image) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:11.711887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.713252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.718035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 523/1043] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:11.723774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.785952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:11.799624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcca5fef-a409-46b3-b70f-cc5426c59319 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.830481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba0703f-7187-47b3-a04d-fcc070b522b0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:11.838092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.838744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.839025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.839691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.848105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.849565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:10:11.861290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:11.862186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:11.863085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 521/1044] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:11.868146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:11.868257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:11.868771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:11.869053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:11.869326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:11.870031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:11.872423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:11.907544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 524/1045] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:10:11 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:11.989015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd386a3-f0ae-4d39-9d99-6b46977f60ad None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:12.031197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:12.031848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.032160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.032434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.033194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:12.033495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:12.034925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c8ce79-7f4d-4596-ad89-305f0b6e2bcc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:12.047543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 522/1046] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:12.053891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:12.084491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dea643-e5ef-4f2e-aee7-bb639804fe6e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.084805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dea643-e5ef-4f2e-aee7-bb639804fe6e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.085058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dea643-e5ef-4f2e-aee7-bb639804fe6e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.086554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 523/1047] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:12.091338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.091669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.091946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.099132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:12.110828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:12.112284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:12.113556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:12.178246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b606c0-13e9-4aaa-9dee-c4d35632026d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:12.191592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 524/1048] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:12.212180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:12.212729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:12.212855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.213067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.213342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.214140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] RBAC: Adding query filter params (role.id=8b7c08cb494340d58b4cb2b6d425c5c0, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:12.214451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:12.215921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:12.221964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 525/1049] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:12 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=8b7c08cb494340d58b4cb2b6d425c5c0 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:12.329131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b9e1b0-1901-420b-93e3-b501ab37a647 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:12.372129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:12.372829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.373118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.373394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.374562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:12.376076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf326bc-3dc4-4df2-ab89-c235c214b7f5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:12.377685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 525/1050] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:12.383318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:12.483309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d331b30d-5148-4ef5-8775-fd6529faf1e8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:12.496484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:12.518351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:12.518971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:12.519254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:12.519558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:12.520251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:12.521690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f1db2-06b9-4673-89c6-01e56ccb11f2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:12.543852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 526/1051] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:10:12 2022] POST /identity/v3/users => generated 285 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:13.954486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173ea-6046-4cf1-9b83-3fc3adcf4373 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:13.955018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173ea-6046-4cf1-9b83-3fc3adcf4373 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:13.955018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6173ea-6046-4cf1-9b83-3fc3adcf4373 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:13.956449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 526/1052] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:13.961486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b700af-ca7e-4d8f-b506-f50faa984ef1 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:13.961885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b700af-ca7e-4d8f-b506-f50faa984ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:13.962044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b700af-ca7e-4d8f-b506-f50faa984ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:13.980392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d1b700af-ca7e-4d8f-b506-f50faa984ef1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:13.981569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d1b700af-ca7e-4d8f-b506-f50faa984ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:14.046865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 527/1053] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:14.069463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173ea-6046-4cf1-9b83-3fc3adcf4373 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.072364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.176219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6173ea-6046-4cf1-9b83-3fc3adcf4373 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.194957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274e551-cd48-4632-b07a-ea6370e74629 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.195239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274e551-cd48-4632-b07a-ea6370e74629 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.195551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274e551-cd48-4632-b07a-ea6370e74629 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.197054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 528/1054] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:14.201825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.202115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.202388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.212220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.212782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.213063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.213338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.222130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:14.223330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:14.286166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:14.287643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:14.289493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.291150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 529/1055] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:14.313597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.343534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.394525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 527/1056] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:14.400173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.424854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aecdf33-fda6-41c8-b961-9d54327aac0e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.462090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.462806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.463025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.463300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.473943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:14.478868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:14.480027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:14.480876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 530/1057] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:14.486015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.507521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e5613-786a-4d11-b86d-e16a09aa6949 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.509575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.546474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.547150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.547450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.547791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.558398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:14.559898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:14.576903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:14.577835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:14.578727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:14.578727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:14.578727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:14.578727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:14.578727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 528/1058] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:14.580635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.586174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.597325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23edd011-6b86-4688-8825-ff6435e0868e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.631679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.632299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.632576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.632847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.633568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:14.633868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:14.635300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43c0de9-4a6f-4ad0-b8ec-97b206307157 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:14.644042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 531/1059] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:14.645296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b644a-bfc7-43d6-a71e-bf7a2ea42bc8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.645585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b644a-bfc7-43d6-a71e-bf7a2ea42bc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.645861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090b644a-bfc7-43d6-a71e-bf7a2ea42bc8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.647064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 532/1060] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:14.649414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b644a-bfc7-43d6-a71e-bf7a2ea42bc8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.692112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3985ad0f-8005-46bf-8290-42a929e976a9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.725548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.726100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.726384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.726657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.727350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:14.727682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:14.729029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7216924b-6871-49b4-87d5-cc3c6be20201 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:14.740691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 529/1061] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:14.741872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.742165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.742442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.760675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090b644a-bfc7-43d6-a71e-bf7a2ea42bc8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.763281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:14.764582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:14.776478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.800920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:14.801651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:14.802012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:14.802344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:14.817102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:14.819240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:14.836031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:14.837030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:14.837943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:14.837943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:14.837943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:14.837943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:14.837943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 533/1062] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:14.843273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.846271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:14.849944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 530/1063] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:14.874019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:14.967316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7339f9-7131-43f3-b52c-301c4ab50117 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:14.983512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba7bf8-5b56-445e-8643-2bb20c86b711 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.004794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.005518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.005872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.006143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.006849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.007148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.008552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.022242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 534/1064] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.022989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.023836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.024115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.024385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.027893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.036232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:15.041070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:15.042189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:15.043229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 531/1065] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:15.048808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.053099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.093040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.135394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616e6c35-ffd2-47b0-9e23-a9c1dfff5e33 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.153802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13a34f7-0a11-4cf7-ad9d-b8cbda6b0df7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.169138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.169781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.170062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.170336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.178083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.179558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:10:15.191616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:15.192566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:15.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 535/1066] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:15.195501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.196159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.196444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.196723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.197447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.197848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.198763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.199212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.212208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 532/1067] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.213297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.277307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.299549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc383354-9813-44c8-9263-b52d823c83f1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.314502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16c4008-65e4-4285-b440-dd78d508bb8c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.321625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.333502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.334166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.334443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.334712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.335520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.335821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.337223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.347857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.348495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.348774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.349050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] PATH_INFO: `/v3/users/00a4054e6ff6468e99da9188935d36c6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.354623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 536/1068] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.360893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.362931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] RBAC: Authorizing `identity:get_user(user_id=00a4054e6ff6468e99da9188935d36c6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.364421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.366093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 533/1069] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/users/00a4054e6ff6468e99da9188935d36c6 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:15.371537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.457745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd8a77-f130-4186-bee8-f4f6ddf57eb3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.467181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac68840-97af-46b7-a162-cf7917ffa1f9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.491796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.492482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.492917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.493407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.502360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.505844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.507297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.513409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.514028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.514309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.514585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:15.521765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:15.522641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:15.523572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:15.523572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:15.523572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:15.523572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:15.523572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 537/1070] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:15.528615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.559304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.586795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.588353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.627407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9107a9-cc88-460b-8937-a2b9db793d56 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.660463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.661070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.661344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.661614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.662298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.662601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.664010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.675497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 538/1071] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.680848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.689678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 534/1072] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:10:15.690997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.714964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.768111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.777537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc9da212-352e-42e5-92d2-447d4f8bfe62 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.788495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9864715-036c-4335-82a5-56f5d9032b6a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.810647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.811305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.811610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.811883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.818771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.820261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.821532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.822171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.822452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.822731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.823907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.825313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.826960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 535/1073] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:10:15.832756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:15.833711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:15.834956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:15.835396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.835990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 539/1074] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:15.838899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:15.924197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:15.933684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f06a56-ce1f-465e-b674-85cf9954e1f2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.936642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07dbbf12-5907-48d8-a8a0-f8ef83f37df4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:15.967701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.968322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.968602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.968877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.969699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=a9cbac9d835f4b7a8327a67955597e0c, user.id=00a4054e6ff6468e99da9188935d36c6) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.970012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.970533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:15.971142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:15.971437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:15.971533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.971720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:15.972467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:15.972773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:15.974175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222882c8-4436-4a3e-aad0-848558a06b44 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:15.986558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 536/1075] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/role_assignments?user.id=00a4054e6ff6468e99da9188935d36c6&scope.project.id=a9cbac9d835f4b7a8327a67955597e0c&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:15.987104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 540/1076] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:10:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:15.992690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:16.097331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e89938-70f8-42e0-b904-1ee409095e6a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:16.131371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:16.132066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:16.132352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:16.132621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/8b7c08cb494340d58b4cb2b6d425c5c0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:16.134807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=d04ab058ac79408f80cc485082c6591c, role_id=8b7c08cb494340d58b4cb2b6d425c5c0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:16.136280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eeb51f-de7d-4f54-931e-0563fe1fa34e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:16.153047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 537/1077] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:10:15 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/d04ab058ac79408f80cc485082c6591c/roles/8b7c08cb494340d58b4cb2b6d425c5c0 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:10:17.972411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916bf37-18a8-45ab-8ca4-af8419615ee8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:17.972944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916bf37-18a8-45ab-8ca4-af8419615ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:17.972944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f916bf37-18a8-45ab-8ca4-af8419615ee8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:17.974318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 541/1078] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:17.979870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bf0a7-83ba-4850-87a8-beb673322923 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:17.980284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bf0a7-83ba-4850-87a8-beb673322923 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:17.980428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9bf0a7-83ba-4850-87a8-beb673322923 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:17.998458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ed9bf0a7-83ba-4850-87a8-beb673322923 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:17.999668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ed9bf0a7-83ba-4850-87a8-beb673322923 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:18.010553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.067144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 538/1079] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:18.089152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916bf37-18a8-45ab-8ca4-af8419615ee8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.103722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.107384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3345-8c22-459a-a51b-3fffc52ba8a7 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.107705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3345-8c22-459a-a51b-3fffc52ba8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.107984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1b3345-8c22-459a-a51b-3fffc52ba8a7 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.109312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 539/1080] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:18.114798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.115110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.115382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.133810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:18.135027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:18.199778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f916bf37-18a8-45ab-8ca4-af8419615ee8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.210480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 540/1081] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:18 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:18.233643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.233958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.234233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.234473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.234572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.245172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:18.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:18.250900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:18.251799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:18.255584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 542/1082] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:18.256890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.309279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.326949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.332095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a758c79-a32a-4b0b-b64f-a99a4b235402 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.356523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a50e0e1-8a77-458a-8cbb-90c913da9d2f None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.365414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.366049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.366325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.366594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.376441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:18.381349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:18.382357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:18.383200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 541/1083] 10.222.0.40 () {56 vars in 1120 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:18.388415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.391699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.392320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.392602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.392880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.393593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:18.393899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.395252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.403291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 543/1084] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:18.409380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.492080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fef928-aa69-4fb7-94cb-c45ccdbf570c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.507384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851d3ad-75d7-4892-a48f-47098559058b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.526831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.527476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.527762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.528038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.528760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:18.529066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.530428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.533310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.539811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 542/1085] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:18.541132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.541766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.542047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.542317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] PATH_INFO: `/v3/users/00a4054e6ff6468e99da9188935d36c6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.543509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] RBAC: Authorizing `identity:get_user(user_id=00a4054e6ff6468e99da9188935d36c6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.545198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.545379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.551999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 544/1086] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/users/00a4054e6ff6468e99da9188935d36c6 => generated 285 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:18.552886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.555575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.647901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ac28565-2e13-4455-80a8-4e848d51feb1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.650630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfb50f7b-8072-4fe4-b66d-97bcc9ba8ce6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.681277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.681940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.682223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.682497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.683751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.684387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.684664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.684929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.686099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.687555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.689871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 545/1087] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:18.694167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.696002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.696151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:18.710176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:18.711074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:18.712040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:18.712040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:18.712040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:18.712040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:18.712040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 543/1088] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:10:18.716886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.765394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.765746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:18.799615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d6541d-6ab5-4551-8b24-b9655b5d3078 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.815205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf509e4-35bd-411c-823f-4ba184e2de66 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:18.832796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.833749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.834014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.834285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c/users/00a4054e6ff6468e99da9188935d36c6/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.836253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cbac9d835f4b7a8327a67955597e0c, user_id=00a4054e6ff6468e99da9188935d36c6, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.837662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.849089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:18.849731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:18.850015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:18.850292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:18.850989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:18.851296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:18.851816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 546/1089] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:10:18 2022] PUT /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c/users/00a4054e6ff6468e99da9188935d36c6/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:10:18.866810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:18.866810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 544/1090] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:18.871771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:18.976239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fca848-25fe-4120-9d2c-e5edb36071b9 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:19.008101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:19.014548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:19.015368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:19.015902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:19.016336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:19.023952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:19.025383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:10:19.036770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:19.037755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:19.038743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 547/1091] 10.222.0.40 () {56 vars in 1093 bytes} [Sun Jul 31 14:10:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:19.043816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:19.142435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8300c904-e7d4-48b5-a326-4f136b990071 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:19.153495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:19.177362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:19.177978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:19.178252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:19.178521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:19.179310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:19.179672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:19.181075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:19.197822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 545/1092] 10.222.0.40 () {56 vars in 1133 bytes} [Sun Jul 31 14:10:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:19.203555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:19.299971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0b571-e3f0-498c-b4c7-956f0ea44f9b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:19.332759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:19.333366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:19.333643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:19.333914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:19.388195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:19.399288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:19.400755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:19.500700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 548/1093] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:19 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:10:19.501970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:19.601825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b01d57a-5b7a-42a3-9403-9381c464c42f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:19.635108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:19.635786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:19.636069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:19.636340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:19.646954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:19.648415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:19.663154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:19.664186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:19.665475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:10:19.665475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:19.665475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:10:19.665475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:19.665475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 546/1094] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:19.670033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:19.783192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:19.794243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07558cf9-83e2-462b-a61b-0bf0caa4e5c4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:19.830567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:19.831184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:19.831490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:19.831779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:19.832469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:19.832945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:19.834605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:19.846805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 549/1095] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:19.852869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:19.883602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:19.973302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6016695-dd46-4e66-b924-802f51c9638f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:20.007737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:20.008375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:20.008679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:20.008952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:20.019332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:20.021028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:10:20.035236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:20.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:20.037067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 547/1096] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:20.042369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:20.144971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6d89a3-9be4-4855-9ffe-65b58f9f7972 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:20.177872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877f033-a530-4a08-b883-94881848c579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:20.178486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f033-a530-4a08-b883-94881848c579 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:20.178760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f033-a530-4a08-b883-94881848c579 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:20.179026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877f033-a530-4a08-b883-94881848c579 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:20.179798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f033-a530-4a08-b883-94881848c579 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:20.180102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f033-a530-4a08-b883-94881848c579 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:20.181515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877f033-a530-4a08-b883-94881848c579 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:20.193031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 550/1097] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:10:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:20.199372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:20.277089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:20.296422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5d143a-2e98-4db8-a351-97d7eb00ad01 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:20.329038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:20.329664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:20.329941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:20.330209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:20.331019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] RBAC: Adding query filter params (role.id=8b7c08cb494340d58b4cb2b6d425c5c0, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=d04ab058ac79408f80cc485082c6591c) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:20.331323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:20.332750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b37a3aa-fec9-4ab5-955f-ecb1c08efe48 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:20.338385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 548/1098] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:20 2022] GET /identity/v3/role_assignments?user.id=d04ab058ac79408f80cc485082c6591c&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=8b7c08cb494340d58b4cb2b6d425c5c0 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:20.877633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd7a7-729d-4178-bb85-023c26f2be65 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:20.878048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd7a7-729d-4178-bb85-023c26f2be65 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:20.878325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9bd7a7-729d-4178-bb85-023c26f2be65 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:20.879750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 551/1099] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:20.885295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:20.885584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:20.885859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:20.903859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:20.905035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:20.978801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 549/1100] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:21.002890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bd7a7-729d-4178-bb85-023c26f2be65 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.015728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:21.108482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9bd7a7-729d-4178-bb85-023c26f2be65 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:21.145890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:21.146555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:21.146837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:21.147112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:21.158360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:21.163219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:21.164283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:21.165384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 552/1101] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:21.170548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.248019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:21.271517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e7272f-28bb-4255-83b6-2ce7f4ec11cb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:21.303973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:21.304579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:21.304857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:21.305127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:21.305838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:21.306139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:21.307511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:21.315284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 550/1102] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:21.322146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.389154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:21.419017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0126af33-8eb9-432e-983e-bf28c448e781 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:21.451815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:21.452442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:21.452724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:21.453003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] PATH_INFO: `/v3/users/00a4054e6ff6468e99da9188935d36c6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:21.454146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] RBAC: Authorizing `identity:get_user(user_id=00a4054e6ff6468e99da9188935d36c6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:21.455590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:21.457175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 553/1103] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/users/00a4054e6ff6468e99da9188935d36c6 => generated 285 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:21.462787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.563505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0c6510-924c-47fb-9ebf-0ef8d17ffc1d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:21.596195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:21.596812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:21.597095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:21.597370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:21.600693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:21.662543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:21.664025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:21.756346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 551/1104] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:21.762444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.863537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c6ee22-a698-49da-935b-92f18c38103d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:21.896167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:21.896793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:21.897068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:21.897337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:21.898463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:21.899922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6986a8-392b-4553-bf93-e620a1a5eafe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:21.901573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 554/1105] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:21.908335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:21.941091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:22.002486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68602ba3-4c98-4e40-a158-0ae6d40bf65c None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:22.035059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:22.035624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:22.035907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:22.036183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:22.036973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=a9cbac9d835f4b7a8327a67955597e0c, user.id=00a4054e6ff6468e99da9188935d36c6) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:22.037284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:22.038700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ceb528b-10f0-4dcf-98ce-bdada7352100 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:22.044342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 552/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:21 2022] GET /identity/v3/role_assignments?user.id=00a4054e6ff6468e99da9188935d36c6&scope.project.id=a9cbac9d835f4b7a8327a67955597e0c&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:24.098254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558ae71-8717-4902-8ec1-c75d62e31aa1 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:24.098806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558ae71-8717-4902-8ec1-c75d62e31aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:24.098806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f558ae71-8717-4902-8ec1-c75d62e31aa1 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:24.100184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 555/1107] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:24.105307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:24.105689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:24.105863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:24.124004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:24.126097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:24.190788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 553/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:24.215060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558ae71-8717-4902-8ec1-c75d62e31aa1 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:24.224619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:24.312987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f558ae71-8717-4902-8ec1-c75d62e31aa1 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:24.345184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:24.345801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:24.346075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:24.346340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:24.347479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:24.348938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b8c008-fe92-4b84-b101-d661fedaeb13 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:24.350439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 556/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:24.356182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:24.385247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:24.451964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2237cc5-04d3-499b-ab2a-ab32f13d3f72 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:24.484376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:24.485576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:24.485858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:24.486137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:24.486822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:24.488261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46506d3-86b6-40ac-9e34-5cf65f8c3fb7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:24.508863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 554/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:10:24 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:26.536073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363d3b9-8202-49fc-b165-0a66e5b97bf3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:26.536573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363d3b9-8202-49fc-b165-0a66e5b97bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:26.536573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8363d3b9-8202-49fc-b165-0a66e5b97bf3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:26.537893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 557/1111] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:26.543074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:26.543466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:26.543656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:26.561716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:26.562925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:26.625999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 555/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:26.648486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363d3b9-8202-49fc-b165-0a66e5b97bf3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:26.722079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:26.747533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8363d3b9-8202-49fc-b165-0a66e5b97bf3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:26.780030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:26.780657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:26.780936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:26.781210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] PATH_INFO: `/v3/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:26.783166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] RBAC: Authorizing `identity:get_role(role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:26.784552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:26.785866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 558/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:26 2022] GET /identity/v3/roles/cce34632ed81427db845cc21f77efac0 => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:26.791565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:26.810379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:26.890460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158c0d1a-2eac-402b-939e-17438f5191b3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:26.930418 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:26.931299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:26.931610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:26.931886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] PATH_INFO: `/v3/users/887fd96c731748f0ac21c3d027927828` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:26.944775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] RBAC: Authorizing `identity:get_user(user_id=887fd96c731748f0ac21c3d027927828)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:26.946195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:26.947848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 556/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:26 2022] GET /identity/v3/users/887fd96c731748f0ac21c3d027927828 => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:26.954110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:27.049376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8dcfec-dd03-47d6-8562-5002bf6b0a78 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:27.081899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:27.082496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:27.082776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:27.083048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:27.094812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:27.148087 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:27.149530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:27.247709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 559/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:27.254027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:27.324618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:27.349156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ded6d0d-c5f0-4189-81ce-e00fec29902a None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:27.381103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:27.381729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:27.382001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:27.382263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:27.383390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:27.384815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c8f6f-64a2-4ed1-b726-4a65b67a8a59 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:27.386436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 557/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:27 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:27.393159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:27.433614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:27.489306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74193e4-6f62-4635-9b9d-a3449a4529fe None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:27.522503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:27.523110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:27.523392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:27.523696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:27.524477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=a9cbac9d835f4b7a8327a67955597e0c, user.id=887fd96c731748f0ac21c3d027927828) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:27.524789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:27.526212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24fdc079-9d33-404c-abd2-5e26c6a5c7ff None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:27.531640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 560/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:27 2022] GET /identity/v3/role_assignments?user.id=887fd96c731748f0ac21c3d027927828&scope.project.id=a9cbac9d835f4b7a8327a67955597e0c&role.id=cce34632ed81427db845cc21f77efac0 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:29.497262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a2f2-5119-4648-93e6-b9f9bbc29869 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:29.497776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a2f2-5119-4648-93e6-b9f9bbc29869 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:29.497776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a2f2-5119-4648-93e6-b9f9bbc29869 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:29.499068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 558/1118] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:29.504300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:29.504717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:29.504855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:29.522426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:29.523642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:29.587956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 561/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:29.611222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5a2f2-5119-4648-93e6-b9f9bbc29869 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:29.707438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5a2f2-5119-4648-93e6-b9f9bbc29869 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:29.740193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:29.740817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:29.741097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:29.741373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] PATH_INFO: `/v3/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:29.742791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] RBAC: Authorizing `identity:get_role(role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:29.744168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:29.745475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 559/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:29 2022] GET /identity/v3/roles/cce34632ed81427db845cc21f77efac0 => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:29.751265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:29.784269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:29.846967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fd7cb5-fb47-4b13-81cb-dd526a441669 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:29.878657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:29.879291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:29.879603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:29.879875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] PATH_INFO: `/v3/users/887fd96c731748f0ac21c3d027927828` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:29.881004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] RBAC: Authorizing `identity:get_user(user_id=887fd96c731748f0ac21c3d027927828)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:29.882417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:29.884025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 562/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:29 2022] GET /identity/v3/users/887fd96c731748f0ac21c3d027927828 => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:29.889722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:29.902454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:29.986529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c207039e-8d0f-49eb-bc46-97e492e9c378 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:30.018714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:30.019287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:30.019597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:30.019872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:30.021000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:30.022408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c08f2bf-3c76-456a-9dce-99c85be93a5a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:30.024066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 560/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:29 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:30.029790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:30.126553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632390b-88e9-4ff9-9a94-cf8c6a14b5b7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:30.128867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:30.160024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:30.160682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:30.160959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:30.161233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c/users/887fd96c731748f0ac21c3d027927828/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:30.163350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a9cbac9d835f4b7a8327a67955597e0c, user_id=887fd96c731748f0ac21c3d027927828, role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:30.164790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ab089f-a15a-41e3-9fe5-22f954c03ef3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:30.176575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 563/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:10:30 2022] PUT /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c/users/887fd96c731748f0ac21c3d027927828/roles/cce34632ed81427db845cc21f77efac0 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:10:32.129140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230a57-559f-4f3b-b292-d4a7dbfdc5c3 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.129686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230a57-559f-4f3b-b292-d4a7dbfdc5c3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.129686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8230a57-559f-4f3b-b292-d4a7dbfdc5c3 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.131014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 561/1124] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:32.136335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.136761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.136894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.154716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:32.155936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:32.220888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 564/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:32.244654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8230a57-559f-4f3b-b292-d4a7dbfdc5c3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:32.320989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:32.342690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8230a57-559f-4f3b-b292-d4a7dbfdc5c3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:32.374393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:32.375639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.375916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.376195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] PATH_INFO: `/v3/roles/cce34632ed81427db845cc21f77efac0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.377606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] RBAC: Authorizing `identity:get_role(role_id=cce34632ed81427db845cc21f77efac0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:32.378948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:32.380339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 562/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:32 2022] GET /identity/v3/roles/cce34632ed81427db845cc21f77efac0 => generated 223 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:32.386025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:32.474079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:32.481418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310879bf-de3c-453a-b7ab-40550860a031 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:32.514534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:32.515164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.515464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.515741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] PATH_INFO: `/v3/users/887fd96c731748f0ac21c3d027927828` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.516891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] RBAC: Authorizing `identity:get_user(user_id=887fd96c731748f0ac21c3d027927828)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:32.518341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:32.519955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 565/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:32 2022] GET /identity/v3/users/887fd96c731748f0ac21c3d027927828 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:32.525732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:32.621563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a2ee74-5dcc-445d-8935-0bd8cf75c337 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:32.652422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:32.652965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.653240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.653510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.668153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:32.718307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:32.720278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:32.813172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 563/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:32.819168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:32.919344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595b4ad-2844-43ab-ac54-2e236467c717 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:32.951779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:32.952409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:32.952689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:32.953482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] PATH_INFO: `/v3/projects/a9cbac9d835f4b7a8327a67955597e0c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:32.954625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] RBAC: Authorizing `identity:get_project(project_id=a9cbac9d835f4b7a8327a67955597e0c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:32.956094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2fc1a9-172f-443b-9ab9-3b842d645515 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:32.957740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 566/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:32 2022] GET /identity/v3/projects/a9cbac9d835f4b7a8327a67955597e0c => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:32.963851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:33.007882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:33.058179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8f2e11-9364-47d4-bc61-9b805f7e1c43 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:33.090236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:33.090830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:33.091104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:33.091371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:33.092177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] RBAC: Adding query filter params (role.id=cce34632ed81427db845cc21f77efac0, scope.project.id=a9cbac9d835f4b7a8327a67955597e0c, user.id=887fd96c731748f0ac21c3d027927828) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:33.092480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:33.094477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c35562-b6c1-4a48-acae-1126ad82ef00 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:33.100082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 564/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:32 2022] GET /identity/v3/role_assignments?user.id=887fd96c731748f0ac21c3d027927828&scope.project.id=a9cbac9d835f4b7a8327a67955597e0c&role.id=cce34632ed81427db845cc21f77efac0 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:35.111025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741ae4-8913-4461-b84d-b639ed67697a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:35.111615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741ae4-8913-4461-b84d-b639ed67697a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:35.111615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc741ae4-8913-4461-b84d-b639ed67697a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:35.112980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 567/1131] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:35.118175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:35.118624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:35.118725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:35.137150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:35.138340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:35.202760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 565/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:35.225283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc741ae4-8913-4461-b84d-b639ed67697a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:35.255642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:35.332828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc741ae4-8913-4461-b84d-b639ed67697a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:35.365435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:35.366008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:35.366288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:35.366562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:35.367734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:35.369163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f034f164-dbd0-41b5-b8d3-5684bec91fe0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:35.370745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 568/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:35 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:35.377241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:35.422306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:35.477864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f67eb3c-9ae4-4732-b158-66a061f7235f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:35.510174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:35.510792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:35.511068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:35.511342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:35.512046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:35.513472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d7900c-e9fa-4a65-ad9d-8e6cf4da9a3e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:35.539514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 566/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:10:35 2022] POST /identity/v3/projects => generated 311 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:37.554128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b13e1-dc77-4410-8415-e5529f7e15b4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:37.554725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b13e1-dc77-4410-8415-e5529f7e15b4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:37.554725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934b13e1-dc77-4410-8415-e5529f7e15b4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:37.555984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 569/1135] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:37.561647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:37.562091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:37.562183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:37.584136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:37.585341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:37.665569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 567/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:37.689937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b13e1-dc77-4410-8415-e5529f7e15b4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:37.783229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:37.787885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934b13e1-dc77-4410-8415-e5529f7e15b4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:37.821106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:37.821674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:37.821953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:37.822227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:37.823382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:37.824821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc051686-05e4-4b08-ad5a-1d70c64f8632 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:37.826382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 570/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:10:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:37.832643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:37.855319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:37.932063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516b572-b717-4a43-b754-4f3ae5f548c8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:37.976892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:37.977494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:37.977768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:37.978041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:37.978726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:37.980163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa414054-0f2e-4e45-bef3-e53657d64e93 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:38.001464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 568/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:10:37 2022] POST /identity/v3/users => generated 286 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:40.054244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef59d5-debc-4142-ac7e-a47ba69dbac2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.054775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef59d5-debc-4142-ac7e-a47ba69dbac2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.054775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ef59d5-debc-4142-ac7e-a47ba69dbac2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.056200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 571/1139] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:40.061317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.061749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.062382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.080779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:40.081957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:40.145930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 569/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:40.168667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef59d5-debc-4142-ac7e-a47ba69dbac2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:40.266632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ef59d5-debc-4142-ac7e-a47ba69dbac2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:40.298098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:40.298716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.298992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.299262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.308971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:40.314212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:40.315377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:40.316382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 572/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:40.321458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:40.331820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:40.414948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f225c81-957e-43ad-b8bc-11294776100c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:40.447977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:40.448584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.448865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.449141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.449858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:40.450163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:40.451674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:40.459437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 570/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:40.465096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:40.492545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:40.562745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfe536e-baec-42d6-b18c-71c398908880 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:40.594437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:40.595062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.595341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.595656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] PATH_INFO: `/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.608077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0e5ef6cc4d433b82726ad1b30b53c2)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:40.610006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:40.611633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 573/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:40 2022] GET /identity/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2 => generated 286 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:40.617395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:40.683845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:40.714423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c42b51-efcc-468c-8997-183d9fb07016 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:40.746749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:40.747354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:40.747668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:40.747948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:40.813206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:40.814574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:40.878694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:40.909379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 571/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:40.915142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:40.995030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:41.010343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b494e45f-bb93-4a64-8690-73e7189aec78 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:41.042646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:41.043274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:41.043582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:41.043858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] PATH_INFO: `/v3/projects/924fe891579c450f837b8a4f7323339c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:41.044997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] RBAC: Authorizing `identity:get_project(project_id=924fe891579c450f837b8a4f7323339c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:41.046999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5a09f82-5e7e-439e-b783-7ae9c937791e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:41.048883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 574/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:40 2022] GET /identity/v3/projects/924fe891579c450f837b8a4f7323339c => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:41.055153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:41.150874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82117e2-3203-41f2-aa05-0e2ca869e8f2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:41.183246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:41.183869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:41.184148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:41.184420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:41.185202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=924fe891579c450f837b8a4f7323339c, user.id=cf0e5ef6cc4d433b82726ad1b30b53c2) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:41.185507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:41.186936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115c9921-aaa8-432a-a4e9-3fa033ace770 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:41.192150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 572/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:41 2022] GET /identity/v3/role_assignments?user.id=cf0e5ef6cc4d433b82726ad1b30b53c2&scope.project.id=924fe891579c450f837b8a4f7323339c&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:43.160097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf319-b516-4ee5-a10c-bb4af2a0b7a6 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.160517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf319-b516-4ee5-a10c-bb4af2a0b7a6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.160656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cecf319-b516-4ee5-a10c-bb4af2a0b7a6 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.162017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 575/1147] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:43.167089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.167570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.167662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.185104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:43.186262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:43.212583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:43.250963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 573/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:43.273638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf319-b516-4ee5-a10c-bb4af2a0b7a6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:43.371372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cecf319-b516-4ee5-a10c-bb4af2a0b7a6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:43.404026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:43.404595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.404876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.405148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.414872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:43.419709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:43.420812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:43.422362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 576/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 148 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:43.427082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:43.521213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c451323-217e-42a6-b091-f7b2593e527a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:43.553020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:43.553612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.553888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.554157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.554862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:43.555158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:43.556531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:43.564741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 574/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:43.570664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:43.585358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:43.666578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7829b0a-949b-4ab9-be71-1d734f124338 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:43.700058 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:43.700679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.700958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.701232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] PATH_INFO: `/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.702364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0e5ef6cc4d433b82726ad1b30b53c2)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:43.703824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:43.705381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 577/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:43 2022] GET /identity/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:43.711138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:43.738407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:43.806363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ac0d4e-ddc7-4281-a07d-2a6b5f1433db None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:43.837542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:43.838158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.838434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.838704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] PATH_INFO: `/v3/projects/924fe891579c450f837b8a4f7323339c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.839842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] RBAC: Authorizing `identity:get_project(project_id=924fe891579c450f837b8a4f7323339c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:43.841268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e6aeef-991e-47bc-8359-8e89a1aa6691 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:43.842876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 575/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:43 2022] GET /identity/v3/projects/924fe891579c450f837b8a4f7323339c => generated 311 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:43.849200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:43.925907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:43.944171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee5336-327a-441a-9c9f-1742a8603753 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:43.976300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:43.976946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:43.977221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:43.977492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] PATH_INFO: `/v3/projects/924fe891579c450f837b8a4f7323339c/users/cf0e5ef6cc4d433b82726ad1b30b53c2/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:43.979384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=924fe891579c450f837b8a4f7323339c, user_id=cf0e5ef6cc4d433b82726ad1b30b53c2, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:43.980818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-933a022f-8de2-406f-895e-1f24813013c5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:43.993880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 578/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:10:43 2022] PUT /identity/v3/projects/924fe891579c450f837b8a4f7323339c/users/cf0e5ef6cc4d433b82726ad1b30b53c2/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:10:45.974450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b7d7-21e1-43b9-bde3-eb19612414a5 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:45.974987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b7d7-21e1-43b9-bde3-eb19612414a5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:45.974987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e42b7d7-21e1-43b9-bde3-eb19612414a5 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:45.976390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 576/1154] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:45.981404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:45.981827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:45.981953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:45.999698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:46.001373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:46.065047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 579/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:46.088874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42b7d7-21e1-43b9-bde3-eb19612414a5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:46.179876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:46.188330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e42b7d7-21e1-43b9-bde3-eb19612414a5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:46.220970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:46.221602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:46.221884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:46.222162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:46.231919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:46.236772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:46.237855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:46.238761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 577/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:46.244034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:46.309244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:46.339542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0155e2-62e8-4dd3-a82f-e33f55c679b5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:46.371693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:46.372294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:46.372571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:46.372842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:46.373550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:46.373850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:46.376004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:46.383719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 580/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:46.389438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:46.485323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fda882-7f18-4d05-b370-eb995dc0d16b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:46.517711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:46.518330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:46.518610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:46.518883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] PATH_INFO: `/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:46.520068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0e5ef6cc4d433b82726ad1b30b53c2)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:46.521471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:46.523042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 578/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/users/cf0e5ef6cc4d433b82726ad1b30b53c2 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:46.528546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:46.623664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14884a0b-acb0-4ac0-9ac3-3af99f1664a3 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:46.651375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:46.656754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:46.657361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:46.657643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:46.657920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:46.723008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:46.724473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:46.819530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 581/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:46.825260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:46.849357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:46.921218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f351e0-ad30-42e9-a8da-1fefc613eed4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:46.952973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:46.953594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:46.953870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:46.954140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] PATH_INFO: `/v3/projects/924fe891579c450f837b8a4f7323339c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:46.955258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] RBAC: Authorizing `identity:get_project(project_id=924fe891579c450f837b8a4f7323339c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:46.956698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d0857b-ac36-4317-97cf-19b2ed294d06 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:46.958300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 579/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/projects/924fe891579c450f837b8a4f7323339c => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:46.964300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:47.002575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:47.069671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0110b96e-19c4-40ca-9265-1fec892b6096 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:47.102697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:47.103292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:47.103598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:47.103870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:47.104643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=924fe891579c450f837b8a4f7323339c, user.id=cf0e5ef6cc4d433b82726ad1b30b53c2) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:47.104948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:47.106359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d45d9f-6b63-4cc2-bd55-a635e3199906 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:47.111987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 582/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:10:46 2022] GET /identity/v3/role_assignments?user.id=cf0e5ef6cc4d433b82726ad1b30b53c2&scope.project.id=924fe891579c450f837b8a4f7323339c&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:49.079984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83ba0-a7f9-43c7-ad8e-caeff3712fab None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:49.080542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83ba0-a7f9-43c7-ad8e-caeff3712fab None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:49.080542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c83ba0-a7f9-43c7-ad8e-caeff3712fab None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:49.081869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 580/1162] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:10:49.086994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:49.087403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:49.087575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:49.105006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:49.106167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:49.171026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 583/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:49.193495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c83ba0-a7f9-43c7-ad8e-caeff3712fab None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:49.281932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:49.292037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c83ba0-a7f9-43c7-ad8e-caeff3712fab None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:49.330644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:49.331181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:49.331492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:49.331771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:49.407585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:49.409607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:49.498290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:49.508632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 581/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:49.514613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:49.540160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:49.611035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0367699-fc9d-4027-b4e6-97eb13d4b52e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:49.644357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:49.644991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:49.645271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:49.645544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:49.652801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:49.654264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-69bfdc83-c7c0-4979-9f24-57eff2233eb0 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:49.666301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:49.668545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:10:49.669987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 584/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:10:49 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:49.674268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:49.771329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecda839-e98d-4e91-89cb-076fa8484ce3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:49.804738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:49.805348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:49.805630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:49.805904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:49.806595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:49.806900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:49.808286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1077da-7b18-4216-92d0-cd67e7acd8e9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:49.815258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 582/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Sun Jul 31 14:10:49 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:51.777599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6e465-e437-4f17-afb1-ed6a9889924b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:51.778126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6e465-e437-4f17-afb1-ed6a9889924b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:51.778126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c6e465-e437-4f17-afb1-ed6a9889924b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:51.779667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 585/1167] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:51.784694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecd05-15dd-4539-ba8a-48d8edda4586 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:51.785122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecd05-15dd-4539-ba8a-48d8edda4586 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:51.785240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2ecd05-15dd-4539-ba8a-48d8edda4586 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:51.803894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4c2ecd05-15dd-4539-ba8a-48d8edda4586 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:51.805056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4c2ecd05-15dd-4539-ba8a-48d8edda4586 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:51.851738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:51.868506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 583/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:10:51.891211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6e465-e437-4f17-afb1-ed6a9889924b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:51.976496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:51.996381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c6e465-e437-4f17-afb1-ed6a9889924b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:52.028568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:52.029121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:52.029419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:52.029681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:52.030382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:52.032307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7ac2f6-f62a-45a3-96b0-ab17e1a029e7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:52.048029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 586/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Sun Jul 31 14:10:51 2022] POST /identity/v3/domains => generated 267 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:54.122976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d96b7-8cc0-4374-ad14-77df330a2cd1 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:54.123548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d96b7-8cc0-4374-ad14-77df330a2cd1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:54.123548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3d96b7-8cc0-4374-ad14-77df330a2cd1 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:54.124975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 584/1170] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:54.130626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:54.131030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:54.131191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:54.149083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:54.150271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:54.214831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 587/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:54.240465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d96b7-8cc0-4374-ad14-77df330a2cd1 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:54.339549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3d96b7-8cc0-4374-ad14-77df330a2cd1 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:54.372389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:54.373066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:54.373298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:54.373572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] PATH_INFO: `/v3/domains/db308f0535ad4d539fff9217b4ca656d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:54.383599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] RBAC: Authorizing `identity:get_domain(domain_id=db308f0535ad4d539fff9217b4ca656d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:54.385029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd92aa5-a48d-41a5-a346-3f4454693323 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:54.386620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 585/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Jul 31 14:10:54 2022] GET /identity/v3/domains/db308f0535ad4d539fff9217b4ca656d => generated 267 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:54.392546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:54.469641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:54.492911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efdff0-1cdd-4fd4-8966-f697f17326fc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:54.525675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:54.526284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:54.526559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:54.526830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:54.527521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:54.528937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788f0a87-226f-4d5e-9ee2-bf7ac6ec03c8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:54.554345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 588/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:10:54 2022] POST /identity/v3/projects => generated 361 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:10:56.560030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448fdb-610d-4852-97c1-cc05c28473cb None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:56.560537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448fdb-610d-4852-97c1-cc05c28473cb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:56.560537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36448fdb-610d-4852-97c1-cc05c28473cb None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:56.562388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 586/1174] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:56.567883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:56.568260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:56.568436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:56.586498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:56.587694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:56.650590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 589/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:56.674167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448fdb-610d-4852-97c1-cc05c28473cb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:56.720322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:56.773978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36448fdb-610d-4852-97c1-cc05c28473cb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:56.806623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:56.807250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:56.807559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:56.807836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] PATH_INFO: `/v3/domains/db308f0535ad4d539fff9217b4ca656d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:56.808976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=db308f0535ad4d539fff9217b4ca656d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:56.810480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2df6901-a8f8-4c1e-867c-93463a09ff2a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:56.812614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 587/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Sun Jul 31 14:10:56 2022] GET /identity/v3/domains/db308f0535ad4d539fff9217b4ca656d => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:56.818515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:56.872878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:56.913730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30399a92-12d7-4cbc-9cf5-a03aa2544855 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:56.946610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:56.947209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:56.947510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:56.947783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:56.948458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:56.949859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aaf9f8-497a-412e-a497-de7f570bc340 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:56.969815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 590/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:10:56 2022] POST /identity/v3/users => generated 311 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:10:58.998280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36396e-bfe2-4fde-918c-9428aaa8a4eb None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:58.998867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36396e-bfe2-4fde-918c-9428aaa8a4eb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:58.998867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd36396e-bfe2-4fde-918c-9428aaa8a4eb None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.000192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 588/1178] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:10:59.005326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.005772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.005874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.023480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:10:59.024670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:10:59.089868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 591/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:10:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:10:59.113091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36396e-bfe2-4fde-918c-9428aaa8a4eb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:59.210258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd36396e-bfe2-4fde-918c-9428aaa8a4eb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:59.223123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:59.244494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:59.245570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.246072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.246542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.259081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:10:59.263854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:10:59.265508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:10:59.266593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 589/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:10:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:10:59.271879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:59.315651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:59.368438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea53bbfb-0f4e-451e-ad4a-3ad0da7a892c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:59.400892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:59.401503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.401785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.402060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.402770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:10:59.403079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:59.404608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:59.413000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 592/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:10:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:10:59.418727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:59.515758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af27cd-a61a-41d7-b558-ab67944d3072 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:59.547535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:59.548158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.548438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.548710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] PATH_INFO: `/v3/users/b34c1fc2391642159b3b8f8f3a76aab9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.562207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] RBAC: Authorizing `identity:get_user(user_id=b34c1fc2391642159b3b8f8f3a76aab9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:59.563676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:59.565261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 590/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:10:59 2022] GET /identity/v3/users/b34c1fc2391642159b3b8f8f3a76aab9 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:10:59.571197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:59.667512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0956e6d5-dfa3-41f4-b3e2-0e2ca07515d6 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:59.674055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:59.700554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:59.701160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.701443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.701718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.766361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:10:59.767761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:10:59.863815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 593/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:10:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:10:59.869653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:10:59.898855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:10:59.965413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16e081-52c5-4741-a993-1c48e2524d0e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:10:59.996972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:10:59.997588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:10:59.997862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:10:59.998131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] PATH_INFO: `/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:10:59.999855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] RBAC: Authorizing `identity:get_project(project_id=5ab134dcd6ff470ea6c5f87f7b571830)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:00.001267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d407ed-3731-492e-96f9-479fd28d989e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:00.002876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 591/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:10:59 2022] GET /identity/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:00.008831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:00.016698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:00.104356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003310f0-5516-4c2a-a26d-53c3141c7e7e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:00.138310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:00.138924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:00.139203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:00.139514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:00.140304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=5ab134dcd6ff470ea6c5f87f7b571830, user.id=b34c1fc2391642159b3b8f8f3a76aab9) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:11:00.140613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:00.142029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3a4d3-bcf1-4c43-a3b8-55f21da4f5a8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:00.147984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 594/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:11:00 2022] GET /identity/v3/role_assignments?user.id=b34c1fc2391642159b3b8f8f3a76aab9&scope.project.id=5ab134dcd6ff470ea6c5f87f7b571830&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:02.104493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0f63a-dd0c-4fe7-b908-f00d89e40a95 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.105048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0f63a-dd0c-4fe7-b908-f00d89e40a95 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.105048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc0f63a-dd0c-4fe7-b908-f00d89e40a95 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.106388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 592/1186] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:11:02.112082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.112535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.112635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.130774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:11:02.131979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:11:02.191354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:02.196344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 595/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:11:02.218295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0f63a-dd0c-4fe7-b908-f00d89e40a95 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:02.310477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:02.315941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc0f63a-dd0c-4fe7-b908-f00d89e40a95 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:02.348302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:02.348922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.349204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.349476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.363490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:11:02.370374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:11:02.371487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:11:02.372579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 593/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:11:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:11:02.378332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:02.473029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea999b-c0a9-4ed3-9ae9-754c54f33b23 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:02.506159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:02.506771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.507053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.507321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.508060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:11:02.508364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:02.509721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:02.517464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 596/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:11:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:02.523330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:02.618463 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d12ffb2-6bde-45f8-9255-883319df3d21 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:02.650093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:02.650705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.650982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.651251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] PATH_INFO: `/v3/users/b34c1fc2391642159b3b8f8f3a76aab9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.652407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] RBAC: Authorizing `identity:get_user(user_id=b34c1fc2391642159b3b8f8f3a76aab9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:02.653806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:02.655880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 594/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:11:02 2022] GET /identity/v3/users/b34c1fc2391642159b3b8f8f3a76aab9 => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:02.661574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:02.720495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:02.762031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896d60d3-a2cc-40ff-85ff-f8ec0480e674 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:02.794773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:02.795472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.796343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.796675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] PATH_INFO: `/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.797877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] RBAC: Authorizing `identity:get_project(project_id=5ab134dcd6ff470ea6c5f87f7b571830)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:02.799303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fffa5cb-4a94-46bb-a1a8-4c257989a204 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:02.801001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 597/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:11:02 2022] GET /identity/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830 => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:02.807022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:02.834556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:02.903509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3da7c5e-eaa5-4610-aab7-5ef11f0380e6 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:02.935387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:02.936066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:02.936371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:02.936651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] PATH_INFO: `/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830/users/b34c1fc2391642159b3b8f8f3a76aab9/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:02.938577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5ab134dcd6ff470ea6c5f87f7b571830, user_id=b34c1fc2391642159b3b8f8f3a76aab9, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:02.940046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c905783-0ae0-4e9b-b030-ac931753b0e3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:02.952935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 595/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:11:02 2022] PUT /identity/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830/users/b34c1fc2391642159b3b8f8f3a76aab9/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:11:04.898065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8ae3a-8359-49a1-be15-3e1b31402e7f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:04.898619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8ae3a-8359-49a1-be15-3e1b31402e7f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:04.898619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee8ae3a-8359-49a1-be15-3e1b31402e7f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:04.899997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 598/1193] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:11:04.905177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:04.905604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:04.905705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:04.923053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:11:04.924266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:11:04.987273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 596/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:11:05.010248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8ae3a-8359-49a1-be15-3e1b31402e7f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.106969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee8ae3a-8359-49a1-be15-3e1b31402e7f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:05.135262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:05.142614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:05.143723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:05.144216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:05.144909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:05.157783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:11:05.162554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:11:05.163864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:11:05.165123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 599/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:11:05.169809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.188790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:05.268527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce32b67-329f-49d3-9ac6-1c66964bfa51 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:05.300679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:05.301287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:05.301564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:05.301835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:05.302553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:11:05.302851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:05.304254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:05.312538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 597/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:05.318439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.412875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02dec326-fbd6-4f04-919b-af381e07c1a1 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:05.446182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:05.446814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:05.447096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:05.447374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] PATH_INFO: `/v3/users/b34c1fc2391642159b3b8f8f3a76aab9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:05.448555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] RBAC: Authorizing `identity:get_user(user_id=b34c1fc2391642159b3b8f8f3a76aab9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:05.450013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:05.451657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 600/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/users/b34c1fc2391642159b3b8f8f3a76aab9 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:05.457539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.535489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:05.558075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcb656d-51d9-4527-bba7-22352c5aa68f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:05.589833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:05.590444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:05.590727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:05.591002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:05.655717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:05.657152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:05.729990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:05.751946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 598/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:11:05.757908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.814130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:05.855332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f45e793-cda0-4326-acd7-d1736deb678c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:05.886938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:05.887597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:05.887898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:05.888148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] PATH_INFO: `/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:05.889282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=5ab134dcd6ff470ea6c5f87f7b571830)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:05.891183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b3f5fa-ad2a-4a0a-8a40-c061b2793ef2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:05.892843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 601/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/projects/5ab134dcd6ff470ea6c5f87f7b571830 => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:11:05.898984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:11:05.993393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7167bf8-8aac-4c17-aa22-d34cc8c98dd5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:11:06.025700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:11:06.026298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:11:06.026579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:11:06.026853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:11:06.027673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=5ab134dcd6ff470ea6c5f87f7b571830, user.id=b34c1fc2391642159b3b8f8f3a76aab9) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:11:06.027986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:11:06.029390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638252e1-08ef-4117-ac52-2ec623f66362 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:11:06.035687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 599/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:11:05 2022] GET /identity/v3/role_assignments?user.id=b34c1fc2391642159b3b8f8f3a76aab9&scope.project.id=5ab134dcd6ff470ea6c5f87f7b571830&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:11:15.886107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=37 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:11:16.033151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=79 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:00.642146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4d58-87aa-426b-8b0d-7e0210f1aecd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:00.642790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4d58-87aa-426b-8b0d-7e0210f1aecd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:00.642790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2f4d58-87aa-426b-8b0d-7e0210f1aecd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:00.644224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 602/1201] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:00.649681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:00.650141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:00.650234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:00.669135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:00.670427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:00.764203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 600/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:00.781423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f4d58-87aa-426b-8b0d-7e0210f1aecd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:00.916462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2f4d58-87aa-426b-8b0d-7e0210f1aecd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:00.959026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:00.959816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:00.960020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:00.960340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:00.973509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:00.975110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:00.991401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:00.992524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:00.994068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:00.994068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:00.994068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:00.994068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:00.994068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 603/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:00.999987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:01.111840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7a783-b435-4413-a907-65720969b8d7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:01.154259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:01.161347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:01.162586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:01.163329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:01.163620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:01.164480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:01.165491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:01.166326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44177c74-d0dc-43a6-9e27-e418b14075cb None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:01.180445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 601/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:01.186608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:01.254968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:01.334177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af4070f-cf9d-4dd7-ad42-91c87f5237a7 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:01.375415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:01.376063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:01.376361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:01.376604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:01.377299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:01.378724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb52f308-b9ee-4d1a-956e-fe5a0f34fc9a None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:01.408520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 604/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Sun Jul 31 14:12:01 2022] POST /identity/v3/users => generated 251 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:12:03.715174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7bedc-e310-4fe9-8151-8558007ac4da None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:03.715848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7bedc-e310-4fe9-8151-8558007ac4da None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:03.715952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a7bedc-e310-4fe9-8151-8558007ac4da None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:03.717678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 602/1206] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:03.724055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:03.724599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:03.724599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:03.744000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:03.745212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:03.829928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 605/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:03.855413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7bedc-e310-4fe9-8151-8558007ac4da None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:03.980353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a7bedc-e310-4fe9-8151-8558007ac4da None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.019044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.019848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.020135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.020412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.034149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:04.039937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:04.041208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:04.042098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 603/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:12:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:04.047798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:04.092789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:04.167477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005f27e-6959-40dd-9c1a-e8ff5982a469 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.210103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.210729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.211021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.211276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.212049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:04.212356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:04.213872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:04.223236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 606/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:04.229212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:04.286842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:04.343601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8c22e5-8463-443d-8fea-4406dd102d08 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.384097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.384811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.385095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.385396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.397233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:04.398799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:04.419806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:04.421245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:04.422320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:04.422320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:04.422320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:04.422320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:04.425909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 604/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:04.426927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:04.547646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:04.551899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20969e3c-9274-4488-80f6-fceede160621 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.592874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.593555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.593838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.594178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.594975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:04.595324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:04.598646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:04.610917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 607/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:04.616974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:04.735002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be987709-edd1-40a8-acbc-521998812da1 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.736666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:04.774828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.775487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.775765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.776034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.784444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:04.785898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:04.799077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:04.800148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:04.801156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 605/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:04.806555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:04.923243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfe09ff-3766-4080-88e2-78430ea9b501 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:04.961009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:04.961645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:04.961920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:04.962184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:04.962951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:04.963254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:04.964756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:04.982625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 608/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:04.988543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:05.106835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246be81f-c843-4f24-a5b6-e44c928b5b73 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:05.155498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:05.156123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:05.156401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:05.156724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:05.190865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:05.236510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:05.238171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:05.359174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 606/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:12:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:12:05.364808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:05.385933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:05.491623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daaa896-cb6a-4067-bf58-3de3e2744b27 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:05.530321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:05.530991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:05.531307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:05.531631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:05.543993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:05.545470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:05.562690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:05.563869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:05.564879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:05.564879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:05.564879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:05.564879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:05.564879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 609/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:05.570233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:05.645708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:05.693690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4421053a-e139-4f5a-b250-605fd8cb46a0 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:05.733376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:05.734381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:05.734498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:05.734695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:05.735390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:05.736799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:05.737160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:05.757049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 607/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:05.758354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:05.849752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:05.909210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7e74d-cee7-46e4-8c9b-4c7af3fad4c4 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:05.955876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:05.957219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:05.957605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:05.957895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:05.966776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:05.968338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:05.983181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:05.984285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:05.985172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:05.989320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 610/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:05.992130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:06.121738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc3fea-028d-4f96-a20a-fd8fc61a89fc None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:06.155932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:06.169697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:06.170252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:06.170534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:06.170821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:06.171601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:06.171906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:06.173326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c090fcc8-b9a5-45bc-9c90-4f3333bb2b4d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:06.187293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 608/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:06.194059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:06.315926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539ba96-36d4-4017-9c28-9d35d2a602a0 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:06.351435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:06.361351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:06.362191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:06.362548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:06.363060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:06.364201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=dee88b8fbb6645f6a8df6a26c11941af) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:06.364567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:06.366397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b5ea98-c132-4172-8bb8-2a260a263e09 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:06.374661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 611/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:12:06 2022] GET /identity/v3/role_assignments?user.id=dee88b8fbb6645f6a8df6a26c11941af&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 259 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:08.530247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01036d7-9fce-4f1e-affc-72bd104793de None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:08.530776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01036d7-9fce-4f1e-affc-72bd104793de None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:08.530869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01036d7-9fce-4f1e-affc-72bd104793de None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:08.532344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 609/1220] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:08.538124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:08.538575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:08.538829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:08.560605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:08.561951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:08.632593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 612/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:08.655786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01036d7-9fce-4f1e-affc-72bd104793de None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:08.768178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01036d7-9fce-4f1e-affc-72bd104793de None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:08.806791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:08.807459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:08.807839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:08.808634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:08.820538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:08.825680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:08.826821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:08.827812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 610/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:12:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:08.833236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:08.940711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c836eb77-03fa-49f7-872c-c7e87cdaeb23 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:08.966541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:08.979299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:08.980526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:08.980857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:08.981167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:08.981998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] RBAC: Adding query filter params (name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:08.982327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:08.983899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:08.992399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 613/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:12:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:08.999044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:09.046057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:09.109435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683dfba4-586f-43ff-9bf1-c172b242946b None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:09.147865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:09.148530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:09.148913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:09.149198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:09.161473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:09.162992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:09.178400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:09.179457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:09.180786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:09.180786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:09.180786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:09.180786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:09.180786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 611/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:09.185926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:09.295014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d598f5-29b5-43ae-8101-5be61b3bd246 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:09.330978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:09.331703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:09.331897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:09.332164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:09.332876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:09.333176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:09.334537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:09.346725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 614/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:09.352724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:09.453937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:09.463057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eceb263d-f4fe-4fb8-b1af-4ceec7689ec9 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:09.496445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:09.497090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:09.497373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:09.497650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:09.505539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:09.507593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:09.521745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:09.522936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:09.524618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 612/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:09 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:09.529533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:09.583876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:09.641367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564d8651-a4ce-4af1-af3e-15f18d77fdde None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:09.682594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:09.683340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:09.683706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:09.684048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:09.684907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:09.685300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:09.686780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:09.706868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 615/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:09.713529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:09.815461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f3bfa9-7e02-4d13-b5c0-8afc863bc2d8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:09.855441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:09.856161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:09.856469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:09.856831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:09.870393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:09.872033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:09.893020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:09.894114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:09.895013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:09.895013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:09.895013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:09.895013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:09.895013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 613/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:09.900673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:10.010310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25feeef0-b3ad-4139-ac5d-bb10412dd487 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:10.013578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:10.053016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:10.053641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:10.053917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:10.054190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:10.054883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:10.055181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:10.056624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:10.069228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 616/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:10.076958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:10.086627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:10.187999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed23caf6-b5ca-4db6-b2b1-9d8b50e4f6b8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:10.227561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:10.228218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:10.228501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:10.228780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:10.236879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:10.238671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:10.252875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:10.253942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:10.255029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:10.261603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 614/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:10.261739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:10.373507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44af6fe-635f-40db-b312-0147bd7a0e85 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:10.407932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:10.408737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:10.409119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:10.410114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:10.410343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:10.410772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:10.412322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d8032c-1351-4b86-9865-3725fc76e034 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:10.427080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 617/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:10.434101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:10.507304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:10.547663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f7847-7807-4f06-97d8-c728993a1766 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:10.590711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:10.591905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:10.592474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:10.592897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/dee88b8fbb6645f6a8df6a26c11941af/roles/afd48f426e1345ae9fd0de7de4359cd6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:10.610584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=dee88b8fbb6645f6a8df6a26c11941af, role_id=afd48f426e1345ae9fd0de7de4359cd6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:10.612064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b25367-2bd7-43b0-913e-ab555a82df6c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:10.626112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 615/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:12:10 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/dee88b8fbb6645f6a8df6a26c11941af/roles/afd48f426e1345ae9fd0de7de4359cd6 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:12:12.622376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01e11-ebe5-4272-87dd-94736f84e675 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:12.622926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01e11-ebe5-4272-87dd-94736f84e675 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:12.622926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b01e11-ebe5-4272-87dd-94736f84e675 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:12.624306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 618/1233] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:12:12.632884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:12.632884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:12.632884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:12.649283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:12.650629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:12.720608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 616/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:12.743484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b01e11-ebe5-4272-87dd-94736f84e675 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:12.770649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:12.864879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b01e11-ebe5-4272-87dd-94736f84e675 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:12.899827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:12.900409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:12.900694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:12.900971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:12.912934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:12.918203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:12.919215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:12.920294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 619/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Sun Jul 31 14:12:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:12.925401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:12.973862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:13.027983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c866857e-212c-4ad4-9f2f-4d57637fc5f2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.062035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.062648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.062924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.063194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:13.063936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] RBAC: Adding query filter params (name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:13.064229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:13.065575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:13.074704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 617/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Sun Jul 31 14:12:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:13.080321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:13.172693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:13.189843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab5be8-3d24-41b7-a4ce-7d9ed7c8163c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.225477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.226214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.226493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.226763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:13.237245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:13.238689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:13.255910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:13.257036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:13.258032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:13.258032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:13.258032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:13.258032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:13.258032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 620/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:13.263989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:13.362278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d46f9-c63d-487a-8940-84da2260ebbe None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.372028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:13.396165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.396740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.397022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.397295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:13.397984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:13.398289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:13.399690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:13.411460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 618/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:13.417969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:13.523898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1aa9-9b9c-4fc2-bbfa-4014663d2d2b None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.563576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.564274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.564610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.564850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:13.568314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:13.574170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:13.575984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:13.588724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:13.589890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:13.590946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 621/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:13.595962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:13.708580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d99524-d71c-4382-b452-fc45b8729789 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.747358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.748074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.748409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.748677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:13.749480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:13.749786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:13.752385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:13.772065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 619/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:13.778795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:13.897489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab113ca-97b7-4d54-8fbf-9879436cb657 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:13.932269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:13.933343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:13.933923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:13.934522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:14.003144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:14.009996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:14.011502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:14.124836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 622/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:12:13 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:12:14.126147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:14.157631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:14.241459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439b67f7-486f-4538-8c10-8cb06a0e6207 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:14.278393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:14.279033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:14.279313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:14.279620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:14.290185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:14.291807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:14.307942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:14.309065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:14.310056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:14.310056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:14.310056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:14.310056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:14.310056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 620/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:14.315222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:14.416730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:14.423948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca764e-6e25-4a58-afd7-a09d6314e88c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:14.462117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:14.462804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:14.463115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:14.463390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:14.464269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:14.464646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:14.466218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:14.478583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 623/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:14.484443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:14.575113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:14.602692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a65a56d-f73f-4b2c-8185-d3a3b842a91d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:14.641017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:14.641759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:14.642045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:14.642320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:14.650547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:14.652403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:14.665355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:14.666406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:14.667522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 621/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:14.672728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:14.785532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cc1ae5-529f-4dc6-9358-9e95313ddc2e None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:14.828899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:14.829851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:14.830284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:14.830675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:14.831951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:14.832376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:14.834533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510f3aa8-b175-4bf2-aa32-4b35bc7eeb1e None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:14.850867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 624/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:12:14.859195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:14.969488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0979ba-c522-4a93-a7c0-4c2173fbab93 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:15.008433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:15.009056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:15.009344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:15.009619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:15.010415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] RBAC: Adding query filter params (role.id=afd48f426e1345ae9fd0de7de4359cd6, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=dee88b8fbb6645f6a8df6a26c11941af) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:15.010786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:15.012219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4032ab36-4aba-4f53-afc7-213653b0a1dd None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:15.019282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 622/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:12:14 2022] GET /identity/v3/role_assignments?user.id=dee88b8fbb6645f6a8df6a26c11941af&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=afd48f426e1345ae9fd0de7de4359cd6 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:17.054941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e966a-f773-439f-9eaa-ad52082b8bf8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.055683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e966a-f773-439f-9eaa-ad52082b8bf8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.055683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e966a-f773-439f-9eaa-ad52082b8bf8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.056882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 625/1247] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:17.062308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.062744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.062835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.069558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:17.081774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:17.082954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:17.158543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 623/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:17.188943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e966a-f773-439f-9eaa-ad52082b8bf8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:17.212141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:17.309887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e966a-f773-439f-9eaa-ad52082b8bf8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:17.350553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:17.351252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.351649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.352006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.364444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:17.369287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:17.370992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:17.371892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 626/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:12:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:17.378066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:17.500567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9627a1f-01ae-4637-aa5e-0a34309159f2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:17.535988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:17.542990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:17.543758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.544037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.544316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.545140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:17.545484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:17.546986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:17.555663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 624/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:12:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:17.562012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:17.645515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:17.673046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e5c71e-1c72-40d1-8c57-14e32d9bdc38 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:17.713343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:17.714800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.715148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.715604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.729274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:17.731652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:17.753979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:17.755173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:17.761316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:17.761316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:17.761316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:17.761316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:17.761316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 627/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:17.762147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:17.879742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bded8646-98af-4987-867a-e74b6c3b184d None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:17.917160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:17.917867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:17.918156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:17.918431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:17.919148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:17.919492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:17.921084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:17.934672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 625/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:17.940625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:18.050975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5271261f-2d26-474f-819d-d391bb664a89 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:18.079506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:18.089248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:18.089863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:18.090234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:18.090412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:18.098194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:18.099660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:18.111444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:18.112641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:18.113772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 628/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:18.121030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:18.186259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:18.241643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f1e36f-f128-40d2-87d2-c772f4d359d3 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:18.285062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:18.285681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:18.285960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:18.286236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:18.287029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:18.287332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:18.288797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:18.307139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 626/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:18.313490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:18.428004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f88d5d-02f3-407b-b6d8-634a103d1d48 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:18.470032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:18.470653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:18.470925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:18.471199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:18.532365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:18.548917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:18.550504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:18.661072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 629/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:12:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 31 14:12:18.668626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:18.780608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44cd6b9-addd-4a92-b694-971b0d932544 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:18.795624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:18.823744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:18.824529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:18.824842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:18.825172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:18.838250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:18.840107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:18.857862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:18.859197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:18.864442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:18.864442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:18.864442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:18.864442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:18.864442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 627/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:18.866035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:18.962014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:18.975821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c265ac06-dccb-4fce-9c51-e21291cd6711 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:19.012791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:19.013412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:19.013690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:19.013963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:19.014652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:19.014958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:19.016346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:19.033204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 630/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:19.034330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:19.142987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb256f4-e08a-421a-a44d-42c78f8f496e None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:19.182167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:19.182832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:19.183170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:19.183529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:19.190454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:19.192226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:19.205188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:19.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:19.207444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 628/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:19.212567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:19.319134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245f6a54-2b1d-4358-84ca-e0a2e79a2fe5 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:19.358859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:19.359513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:19.359795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:19.360071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:19.360820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:19.361139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:19.362567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37aca7c9-a13f-4524-a8aa-200b99c57f0d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:19.376229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 631/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:19.384613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:19.412748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:19.502223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1d4ba9-45d6-4596-9825-8464ea5a10d7 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:19.539712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:19.540321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:19.540599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:19.540865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:19.541684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=dee88b8fbb6645f6a8df6a26c11941af) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:19.542243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:19.544467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d854518c-a216-4f9e-a756-59e51d1f4f7a None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:19.550695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 629/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:12:19 2022] GET /identity/v3/role_assignments?user.id=dee88b8fbb6645f6a8df6a26c11941af&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:21.615009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96dbe5-d11a-4b27-9d2b-ac83b04f5339 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:21.615546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96dbe5-d11a-4b27-9d2b-ac83b04f5339 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:21.615546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce96dbe5-d11a-4b27-9d2b-ac83b04f5339 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:21.616896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 632/1261] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:21.622483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:21.622901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:21.623028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:21.641294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:21.642483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:21.706380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 630/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:21 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:21.729576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96dbe5-d11a-4b27-9d2b-ac83b04f5339 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:21.740267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:21.841486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce96dbe5-d11a-4b27-9d2b-ac83b04f5339 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:21.881069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:21.881705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:21.881985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:21.882265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:21.894346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:21.899398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:21.900526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:21.901662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 633/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:12:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:21.906552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:21.924626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:22.011885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04af6b-aaa1-4df3-9fce-0c61bcb2dfa8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.059851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.060472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.060754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.061029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.061748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:22.062053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.063411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.071784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 631/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:12:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:22.077649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:22.143864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:22.177747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114c5e58-865b-449c-b0f0-f0429a4ccb1a None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.210797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.211451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.211741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.212016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.222142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.223610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:22.238190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:22.239211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:22.240356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:22.240356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:22.240356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:22.240356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:22.240356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 634/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:22.245284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:22.337970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:22.357612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3814da48-800c-48e0-a218-c18ad260547f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.394340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.394882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.395165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.395467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.396204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:22.396514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.399101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.411634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 632/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:22.417565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:22.518043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852f692b-6e7b-4e9c-a2d2-9fd662f06d77 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.540908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:22.554866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.555526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.555810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.556088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.563361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.564872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:22.578096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:22.579177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:22.580367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 635/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:22.584996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:22.698860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c856735b-db77-473d-8c04-fc76396e5ed4 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.731902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.732495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.732772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.733044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.733827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:22.734131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.735647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.743855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:22.752445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 633/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:22.758128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:22.866216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e07e5a-3b3c-4a65-9627-ea51a00339b2 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:22.899551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:22.900189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:22.900470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:22.900751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:22.910883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:22.912354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:22.926063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:22.927115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:22.928128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:22.928128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:22.928128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:22.928128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:22.928128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 636/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:22.933498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:23.029143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ce2320-086e-4274-918e-71736eb37261 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:23.062104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:23.062708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:23.062989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:23.063263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:23.063988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:23.064293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:23.065642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:23.076987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 634/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:23.082695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:23.113782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:23.180887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7426cdf-0851-40b6-8dbb-e6c6539d9e3d None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:23.213498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:23.214122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:23.214402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:23.214675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:23.221033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:23.222447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:23.233882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:23.234948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:23.236089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 637/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:23.240831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:23.298826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:23.339643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d686c6b6-dc88-4d4e-ab39-581bb67aa0d8 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:23.372224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:23.372833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:23.373115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:23.373385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:23.374134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:23.374460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:23.375925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367b758-464a-459e-84bb-60caf2d873f0 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:23.387366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 635/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:23.393303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:23.485462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:23.494350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa3c6cc-cae8-4ca7-9039-5635b8658bda None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:23.531502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:23.532408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:23.532796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:23.533173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] PATH_INFO: `/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/dee88b8fbb6645f6a8df6a26c11941af/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:23.535757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] RBAC: Authorizing `identity:create_grant(project_id=f9e2337dff494adbaf88dad516c3d98d, user_id=dee88b8fbb6645f6a8df6a26c11941af, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:23.537208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-176e01fd-ae6a-4ab8-b1a4-b4f391a04b81 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:23.552016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 638/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Sun Jul 31 14:12:23 2022] PUT /identity/v3/projects/f9e2337dff494adbaf88dad516c3d98d/users/dee88b8fbb6645f6a8df6a26c11941af/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:12:25.561164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d735b35-f6d9-4d72-a106-77b8afd34a59 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:25.561669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d735b35-f6d9-4d72-a106-77b8afd34a59 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:25.561669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d735b35-f6d9-4d72-a106-77b8afd34a59 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:25.562970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 636/1274] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:12:25.568820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:25.569241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:25.569376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:25.587598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:25.589386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:25.654848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 639/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:25.677900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d735b35-f6d9-4d72-a106-77b8afd34a59 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:25.778407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d735b35-f6d9-4d72-a106-77b8afd34a59 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:25.795140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:25.814892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:25.815814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:25.816504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:25.816878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:25.829889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 31 14:12:25.836295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:25.837549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:25.838598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 637/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Sun Jul 31 14:12:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:25.845329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:25.935598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:25.954577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b13ea4-f2e8-49a7-92fe-47429646ac29 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:25.986312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:25.986924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:25.987203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:25.987503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:25.988214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:25.988517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:25.989870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:25.998156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 640/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Sun Jul 31 14:12:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:26.004530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:26.110093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe487c37-14c7-4eb5-9f34-42f3a0965b03 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:26.143920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:26.144558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:26.144842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:26.145116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:26.154817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:26.156286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:26.171620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:26.172768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:26.173970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:26.173970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:26.173970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:26.173970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:26.173970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 638/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 31 14:12:26.178599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:26.274492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f9fcb5-4732-4d7f-bca4-f3c99b8b39d8 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:26.306476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:26.307692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:26.307977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:26.308251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:26.308937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:26.309243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:26.310591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:26.322025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 641/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:26.327711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:26.366104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:26.437741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f86dc4-dd31-4443-b654-7b70e12021d2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:26.469993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:26.470623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:26.470905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:26.471180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:26.478250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:26.480287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 31 14:12:26.491139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:26.492239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:26.493322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 639/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/users/placement => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:26.498104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:26.501844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:26.593463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b9bbf-dd38-4553-b889-3a61cc32e48c None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:26.626882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:26.627503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:26.627782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:26.628049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:26.628820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:26.629120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:26.630524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:26.646717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 642/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:26.653018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:26.746169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:26.755785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01a5096b-d5c9-48cc-890c-0f9503a01f07 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:26.787829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:26.788435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:26.788714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:26.788990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:26.856951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:26.858386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:26.940813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:26.951645 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 640/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 31 14:12:26.957587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:27.018540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:27.068705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14dbee38-a1f0-4393-990c-8da9eb0a1835 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:27.106405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:27.107035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:27.107309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:27.107612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:27.117057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:27.118476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 31 14:12:27.132811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:27.133908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:27.134946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 31 14:12:27.134946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 31 14:12:27.134946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 31 14:12:27.134946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:27.134946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 643/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:12:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:27.140219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:27.233921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5b771-cc66-4377-82f2-08d146a88a56 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:27.266491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:27.267088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:27.267363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:27.267657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:27.268338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:27.268633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:27.269960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:27.280605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 641/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Sun Jul 31 14:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:27.286159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:27.384880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2853e89-3793-490f-8a5a-378dcfbf19db None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:27.398245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:27.417664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:27.418292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:27.418570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:27.418841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:27.425227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:27.426648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 31 14:12:27.438064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:27.439083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: ERROR keystone.server.flask.application Jul 31 14:12:27.440224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 644/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Sun Jul 31 14:12:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:27.445300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:27.493282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:27.543055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1976bbea-269c-43a6-bd56-f4340af30fd5 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:27.575228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:27.575868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:27.576149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:27.576426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:27.577170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:27.577477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:27.578919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9f2f9f-c08f-4ae6-8b1b-0fe14526dc7c None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:27.590390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 642/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Sun Jul 31 14:12:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:27.596442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:27.692378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab561c6-b599-410b-b2c4-23efa5b06dca None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:27.724947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:27.725546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:27.725823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:27.726095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:27.726874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] RBAC: Adding query filter params (role.id=f459ad5c5a244da98c2cacf1a3be0871, scope.project.id=f9e2337dff494adbaf88dad516c3d98d, user.id=dee88b8fbb6645f6a8df6a26c11941af) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:27.727181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:27.728629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f970f923-e779-4d5c-a3c2-c9c1db5e5be9 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:27.734284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 645/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Sun Jul 31 14:12:27 2022] GET /identity/v3/role_assignments?user.id=dee88b8fbb6645f6a8df6a26c11941af&scope.project.id=f9e2337dff494adbaf88dad516c3d98d&role.id=f459ad5c5a244da98c2cacf1a3be0871 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:12:29.741449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472417e-207b-4683-8ae2-1ae3a4053977 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:29.741977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472417e-207b-4683-8ae2-1ae3a4053977 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:29.741977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6472417e-207b-4683-8ae2-1ae3a4053977 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:29.743323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 643/1288] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:29.749076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:29.749515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:29.749667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:29.767951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:29.769135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:29.798510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:29.841068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 646/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:29.868224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472417e-207b-4683-8ae2-1ae3a4053977 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:29.967878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6472417e-207b-4683-8ae2-1ae3a4053977 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:29.978008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:30.001497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:30.002067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:30.002349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:30.002625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:30.003248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:30.004629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:30.013241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:30.014419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:30.015500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 644/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:12:29 2022] GET /identity/v3/services/placement => generated 90 bytes in 147 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:30.020557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:30.123303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088d5464-a5f1-4292-af02-234f7a7f1d75 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:30.156761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:30.157383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:30.157666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:30.157942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:30.158621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:30.158929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:30.160328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3978a0-e318-42b5-bc4a-81f9c26ab087 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:30.165166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 647/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 31 14:12:30 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:30.171574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:30.268191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95a134-4067-497c-90fe-c9177250125f None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:30.300515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:30.301117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:30.301397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:30.301672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] PATH_INFO: `/v3/services` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:30.302345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:30.302651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:30.304033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5c4f76-d817-48c5-a2ef-65173da01042 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:30.309329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 645/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 31 14:12:30 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:32.266140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a35c8-fe5a-493b-b591-0db48f437c6d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:32.266660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a35c8-fe5a-493b-b591-0db48f437c6d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:32.266772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb6a35c8-fe5a-493b-b591-0db48f437c6d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:32.268059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 648/1293] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:32.273057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980226a-1add-4938-9405-81dcf607e5e2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:32.273491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980226a-1add-4938-9405-81dcf607e5e2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:32.273595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3980226a-1add-4938-9405-81dcf607e5e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:32.291639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3980226a-1add-4938-9405-81dcf607e5e2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:32.292804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3980226a-1add-4938-9405-81dcf607e5e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:32.320485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:32.356588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 646/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 31 14:12:32.378069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a35c8-fe5a-493b-b591-0db48f437c6d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:32.408846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:32.476769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb6a35c8-fe5a-493b-b591-0db48f437c6d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:32.508928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:32.509525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:32.510420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:32.510697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:32.511376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:32.512761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19886927-b2b4-41ac-8c0b-f5d5bf2ed6f8 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:32.526116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 649/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Sun Jul 31 14:12:32 2022] POST /identity/v3/services => generated 246 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:12:34.535232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db74c-10ed-4b8a-8793-f3a98d619197 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:34.535793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db74c-10ed-4b8a-8793-f3a98d619197 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:34.535793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1db74c-10ed-4b8a-8793-f3a98d619197 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:34.537054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 647/1296] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:34.542578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:34.542962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:34.543137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:34.561317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:34.562502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:34.632420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 650/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:34 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:34.654023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db74c-10ed-4b8a-8793-f3a98d619197 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:34.751058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1db74c-10ed-4b8a-8793-f3a98d619197 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:34.783299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:34.783947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:34.784225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:34.784495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:34.785111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:34.786504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:34.796318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:34.797659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:34.798856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 648/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:12:34 2022] GET /identity/v3/services/placement => generated 90 bytes in 144 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:34.803404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:34.822737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:34.917452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2943d48-1806-4bbb-a47f-d6881d890bae None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:34.949693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:34.950307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:34.950588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:34.950862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:34.951565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:34.951873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:34.953223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:34.958207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 651/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 31 14:12:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:34.964866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:34.995920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:35.062031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fc18cfb-ee94-4edf-997f-db1445b3e071 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:35.094286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:35.094877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:35.095154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:35.095451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:35.096198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] RBAC: Adding query filter params (interface=public, service_id=657aba834cf04b61b4a44306c4a9a0ba, region_id=RegionOne) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:35.096587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:35.097933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6119bb1d-308e-4631-bc0b-b4fc2303396e None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:35.102696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 649/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Sun Jul 31 14:12:34 2022] GET /identity/v3/endpoints?service_id=657aba834cf04b61b4a44306c4a9a0ba&interface=public®ion_id=RegionOne => generated 194 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:35.108290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:35.186783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:35.205524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55647445-56bd-44cc-b7f0-e3595b784157 None admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:35.238618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:35.239229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:35.239539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:35.239817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:35.240465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:35.241823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d7501-3612-406e-8593-a7795bacc575 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:35.248542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 652/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Sun Jul 31 14:12:35 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:12:37.223255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bea12-f8e4-4cbd-8759-2d19bacc71a4 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:37.223782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bea12-f8e4-4cbd-8759-2d19bacc71a4 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:37.223941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95bea12-f8e4-4cbd-8759-2d19bacc71a4 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:37.225297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 650/1302] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:12:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:12:37.230843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:37.231280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:37.231381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:37.248942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:12:37.250106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:12:37.312719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 653/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:12:37 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 31 14:12:37.335324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bea12-f8e4-4cbd-8759-2d19bacc71a4 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:37.427488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:37.434318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95bea12-f8e4-4cbd-8759-2d19bacc71a4 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:37.472928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:37.473567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:37.473848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:37.474124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:37.474746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:37.476143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1517, in full_dispatch_request Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1503, in dispatch_request Jul 31 14:12:37.484852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 31 14:12:37.486047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.get_or_create( Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application with Lock( Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self._enter() Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application return self.creator() Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application created_value = creator( Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: ERROR keystone.server.flask.application Jul 31 14:12:37.487104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 651/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Sun Jul 31 14:12:37 2022] GET /identity/v3/services/placement => generated 90 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 31 14:12:37.492160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:37.565017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:37.588233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d4de014-5fdf-4189-bfdd-7c15d004cef9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:37.620081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:37.620689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:37.620968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:37.621241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] PATH_INFO: `/v3/services` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:37.621912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:12:37.622216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:37.623601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe04c6e-8dd3-4be1-b630-ced370caf4f0 None admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:37.628578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 654/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Sun Jul 31 14:12:37 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:12:37.634235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:12:37.731152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938df822-eece-4018-9b9b-71c8349fe8e2 None admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:12:37.763394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:12:37.764019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:12:37.764299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:12:37.764573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:12:37.765280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:12:37.766625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff95d148-1d85-4778-98e3-ddb547229d9b None admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:12:37.785374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 652/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:12:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:12:47.625665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=30 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:12:47.781156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:40.863610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08dcd5-59fc-476c-a869-7be57fa0180f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:40.864234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08dcd5-59fc-476c-a869-7be57fa0180f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:40.864234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc08dcd5-59fc-476c-a869-7be57fa0180f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:40.871139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0a203-cb08-4fd6-8268-67e63e6177bd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:40.871647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0a203-cb08-4fd6-8268-67e63e6177bd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:40.871737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d0a203-cb08-4fd6-8268-67e63e6177bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:40.876258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 655/1307] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:40.957997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-79d0a203-cb08-4fd6-8268-67e63e6177bd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:40.959226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-79d0a203-cb08-4fd6-8268-67e63e6177bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:41.072081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 653/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:13:41.126543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:41.127060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:41.127591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:41.132878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 656/1309] 10.222.0.40 () {54 vars in 965 bytes} [Sun Jul 31 14:13:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:41.142712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9f7f-31c8-4da0-876e-3002da6782b3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:41.143059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9f7f-31c8-4da0-876e-3002da6782b3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:41.143281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fe9f7f-31c8-4da0-876e-3002da6782b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:41.236172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-54fe9f7f-31c8-4da0-876e-3002da6782b3 None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:41.237408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-54fe9f7f-31c8-4da0-876e-3002da6782b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:41.260217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:41.319836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 654/1310] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Jul 31 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:41.332521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:41.335632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:41.460590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e8b074-c081-4050-b5d1-27242e5268ba None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:41.536506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:41.537149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:41.537457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:41.537735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:41.539351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:41.595839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:41.659968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:41.661346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26884761-d23b-4acf-afaa-80f56b68562d service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:41.683501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 657/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:13:44.033497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140ff4-c586-427d-9bbf-80170205473d None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:44.034031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140ff4-c586-427d-9bbf-80170205473d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:44.034146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c140ff4-c586-427d-9bbf-80170205473d None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:44.035675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 655/1312] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:44.041133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3af6b-3f73-4e12-a6b2-29d449726caf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:44.041592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3af6b-3f73-4e12-a6b2-29d449726caf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:44.041672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca3af6b-3f73-4e12-a6b2-29d449726caf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:44.116628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fca3af6b-3f73-4e12-a6b2-29d449726caf None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:44.118650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fca3af6b-3f73-4e12-a6b2-29d449726caf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:44.236849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 658/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:44.289758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:44.289946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:44.290217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:44.291781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 656/1314] 10.222.0.40 () {54 vars in 965 bytes} [Sun Jul 31 14:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:44.298277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5bb7-87f7-408c-a98c-1d78976cfb3d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:44.298575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5bb7-87f7-408c-a98c-1d78976cfb3d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:44.298854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563e5bb7-87f7-408c-a98c-1d78976cfb3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:44.311229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:44.384791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-563e5bb7-87f7-408c-a98c-1d78976cfb3d None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:44.386015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-563e5bb7-87f7-408c-a98c-1d78976cfb3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:44.469406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 659/1315] 10.222.0.40 () {58 vars in 1062 bytes} [Sun Jul 31 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:44.478245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:44.479928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:44.602207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f73a2e-fcf1-4dd6-ac2b-3b3442edeeb5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:44.683211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:44.685781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:44.686617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:44.687053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:44.687629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:44.689262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:44.794499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:44.796015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113a3f25-3e01-44db-8c31-18483a58d7b2 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:44.818277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 657/1316] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:13:47.329889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:47.330454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:47.330454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:47.331819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 660/1317] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:47.337536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb197-d99d-42f5-a5d6-66d0f4fb359b None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:47.337959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb197-d99d-42f5-a5d6-66d0f4fb359b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:47.338083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2bb197-d99d-42f5-a5d6-66d0f4fb359b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:47.402733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5d2bb197-d99d-42f5-a5d6-66d0f4fb359b None None] MFA Rules not processed for user `6692a4b4a7a24f49983927b5b63565f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:47.403899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5d2bb197-d99d-42f5-a5d6-66d0f4fb359b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:47.454025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:47.497388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 658/1318] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:47.536196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:47.537775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:47.639506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4ea4a-d3fe-4d6a-a937-17a0fb2f03ec None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:47.642268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:47.710891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:47.711464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:47.711748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:47.712016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:47.713611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:47.818479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:47.819947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e52362-0867-4b3c-a5b2-5902ac546018 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:47.837748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 661/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:47 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:13:50.389220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:50.389760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:50.389760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:50.391115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 659/1320] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:50.396206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccaa48-df6f-4b5c-88ba-c9ea0f85099e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:50.396608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccaa48-df6f-4b5c-88ba-c9ea0f85099e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:50.396918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbccaa48-df6f-4b5c-88ba-c9ea0f85099e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:50.438541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:50.464425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dbccaa48-df6f-4b5c-88ba-c9ea0f85099e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:50.465639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dbccaa48-df6f-4b5c-88ba-c9ea0f85099e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:50.561763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 662/1321] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:50.600043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:50.601571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:50.702249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010a117a-34a9-4cfc-809a-952d6fb1243e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:50.772619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:50.773236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:50.773514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:50.773786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:50.775404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:50.782680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:50.888268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:50.889709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fed41f-833b-4fd4-b9b5-7b2efee70d34 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:50.908634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 660/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:50 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:13:53.576578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:53.577122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:53.577122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:53.578423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 663/1323] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:53.583707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affcdc9-240f-4546-b5e5-3a9fd429a85d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:53.584159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affcdc9-240f-4546-b5e5-3a9fd429a85d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:53.584416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4affcdc9-240f-4546-b5e5-3a9fd429a85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:53.650969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4affcdc9-240f-4546-b5e5-3a9fd429a85d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:53.652244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4affcdc9-240f-4546-b5e5-3a9fd429a85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:53.712813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:53.755042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 661/1324] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:53.793241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:53.794799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:53.904424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e280e8-2d6a-426d-ba5d-85aef99647e4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:53.916381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:53.978034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:53.978654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:53.978934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:53.979205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:53.980870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:54.089097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:54.090587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3d0c8e-7fae-47b6-b6af-4dc3e0f3062d service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:54.108946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 664/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:13:56.178113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:56.178620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:56.178715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:56.180096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 662/1326] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:56.185408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629451-2b2f-43b2-90df-de4285cb020d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:56.185798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629451-2b2f-43b2-90df-de4285cb020d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:56.185983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc629451-2b2f-43b2-90df-de4285cb020d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:56.233578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:56.252572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fc629451-2b2f-43b2-90df-de4285cb020d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:56.253775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fc629451-2b2f-43b2-90df-de4285cb020d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:56.348226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 665/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:56.385026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:56.386587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:56.490884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2679e1-b1aa-4a20-b3cd-f7d41202ff26 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:56.563088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:56.563771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:56.564050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:56.564321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:56.566181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:56.583098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:56.680696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:56.682123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774c7a83-b37a-4cc8-b859-1d60f6e06092 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:56.700456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 663/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:13:58.777028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:58.777575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:58.777575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:58.778948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 666/1329] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:13:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:13:58.784158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2949e-18f2-494c-859c-4a401d555d77 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:58.784601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2949e-18f2-494c-859c-4a401d555d77 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:58.784707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a2949e-18f2-494c-859c-4a401d555d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:58.848911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-06a2949e-18f2-494c-859c-4a401d555d77 None None] MFA Rules not processed for user `6692a4b4a7a24f49983927b5b63565f7`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:13:58.850087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-06a2949e-18f2-494c-859c-4a401d555d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:58.915325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:58.953126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 664/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:13:58 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:13:58.991062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:13:58.992671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:59.098107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbbb267-c27e-4ccf-a03a-3e97cdce131f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:13:59.118405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:13:59.173570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:13:59.174117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:13:59.174399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:13:59.174677 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:13:59.176332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:13:59.292256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:13:59.294485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6303200b-f545-4292-95b3-617394dfb4cb service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:13:59.319438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 667/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:13:58 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:01.630439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:01.630956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:01.631072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:01.632429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 665/1332] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:01.637572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729858-ab14-49e0-97c0-49d8c45424d3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:01.637955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729858-ab14-49e0-97c0-49d8c45424d3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:01.638128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41729858-ab14-49e0-97c0-49d8c45424d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:01.699277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:01.702068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-41729858-ab14-49e0-97c0-49d8c45424d3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:01.703793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-41729858-ab14-49e0-97c0-49d8c45424d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:01.801505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 668/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:01.837592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:01.839110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:01.939735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8397e7c3-74e0-41fc-a2d1-78b71276fe1c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:02.010508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:02.011118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:02.011398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:02.011709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:02.013329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:02.040459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:02.120055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:02.121476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf012dd1-d00f-4525-ba3e-fdcda114bd6e service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:02.140464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 666/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:04.186678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:04.187251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:04.187370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:04.189231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 669/1335] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:04.195553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183759-e9aa-425a-8967-2878a24c6492 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:04.196003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183759-e9aa-425a-8967-2878a24c6492 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:04.196216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1183759-e9aa-425a-8967-2878a24c6492 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:04.260034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a1183759-e9aa-425a-8967-2878a24c6492 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:04.261227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a1183759-e9aa-425a-8967-2878a24c6492 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:04.333099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:04.356674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 667/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:04.397047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:04.398568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:04.501819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ad83ff-7e9d-4f21-9d1b-e0810dd4d129 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:04.528558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:04.572708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:04.573326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:04.573605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:04.573876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:04.575517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:04.682321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:04.683713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ccf0c2-a16b-4fac-a5c2-98c9b80b6bda service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:04.701770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 670/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:07.049116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:07.049639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:07.049639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:07.050916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 668/1338] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:07.056140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6a8d-f951-4cfe-a29b-17a219d4405f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:07.056539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6a8d-f951-4cfe-a29b-17a219d4405f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:07.056694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cf6a8d-f951-4cfe-a29b-17a219d4405f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:07.121475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c0cf6a8d-f951-4cfe-a29b-17a219d4405f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:07.122707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c0cf6a8d-f951-4cfe-a29b-17a219d4405f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:07.131055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:07.219331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 671/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:07 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:07.255552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:07.257138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:07.476306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997c56c9-5421-4de6-a6f7-e94b7fc706d9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:07.557307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:07.558393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:07.558671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:07.558960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:07.560619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:07.597330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:07.673757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:07.675205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8733af5-ac38-4916-83dd-be2bdc40b132 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:07.693716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 669/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:07 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:14:11.958225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:11.958783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:11.958783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:11.960186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 672/1341] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:11.965444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159c275-bed2-48a7-98e1-6f54e008e9c1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:11.965851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159c275-bed2-48a7-98e1-6f54e008e9c1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:11.965989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1159c275-bed2-48a7-98e1-6f54e008e9c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:12.030657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-1159c275-bed2-48a7-98e1-6f54e008e9c1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:12.031890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1159c275-bed2-48a7-98e1-6f54e008e9c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:12.117787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:12.130386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 670/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:12.168929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:12.170452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:12.273065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb294215-0fbd-4f90-af1b-5456cd7d548d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:12.312005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:12.349545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:12.350166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:12.350444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:12.350714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:12.352360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:12.460817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:12.462298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1b876f-80c9-45b1-803c-848c0f8337c1 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:12.481309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 673/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:15.132271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:15.132814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:15.132814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:15.134194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 671/1344] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:15.140127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea625-dcd0-4de9-939f-19ef408d5ef0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:15.140617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea625-dcd0-4de9-939f-19ef408d5ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:15.140693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4ea625-dcd0-4de9-939f-19ef408d5ef0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:15.215902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2a4ea625-dcd0-4de9-939f-19ef408d5ef0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:15.217165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2a4ea625-dcd0-4de9-939f-19ef408d5ef0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:15.236533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:15.316558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 674/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:15 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:15.353850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:15.355374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:15.457039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f29ad8-e9b6-4285-bfdf-a3eb97bc64bd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:15.541518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99961201-9625-45e8-a694-10197502c416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:15.542070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961201-9625-45e8-a694-10197502c416 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:15.542350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961201-9625-45e8-a694-10197502c416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:15.542621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99961201-9625-45e8-a694-10197502c416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:15.544457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-99961201-9625-45e8-a694-10197502c416 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:15.588188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:15.654524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961201-9625-45e8-a694-10197502c416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:15.656028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99961201-9625-45e8-a694-10197502c416 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:15.676448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 672/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:15 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:19.616939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:19.617482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:19.617482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:19.619008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 675/1347] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:19.624041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dccdfc-9f1d-46ac-bc1f-6d59107d0079 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:19.624458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dccdfc-9f1d-46ac-bc1f-6d59107d0079 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:19.624567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00dccdfc-9f1d-46ac-bc1f-6d59107d0079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:19.694209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-00dccdfc-9f1d-46ac-bc1f-6d59107d0079 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:19.695618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-00dccdfc-9f1d-46ac-bc1f-6d59107d0079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:19.793606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:19.796178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 673/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:19.835097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:19.836679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:19.940566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d029d522-ded2-4d23-bed8-271b8091ce0b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:19.989391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:20.018582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:20.019205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:20.019509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:20.019783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:20.021454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:20.150179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:20.151808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aeabbbf-ddd7-4196-b939-50e6c8cf466c service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:20.172632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 676/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:25.124729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7f646-ba3a-4063-944b-f56c47c3d523 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:25.125863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7f646-ba3a-4063-944b-f56c47c3d523 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:25.126426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb7f646-ba3a-4063-944b-f56c47c3d523 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:25.128283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 674/1350] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:14:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:25.137085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbf7b-b2af-4420-a3d0-041d7f589c76 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:25.137550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbf7b-b2af-4420-a3d0-041d7f589c76 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:25.137646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ccbf7b-b2af-4420-a3d0-041d7f589c76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:25.275262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b1ccbf7b-b2af-4420-a3d0-041d7f589c76 None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:25.276809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b1ccbf7b-b2af-4420-a3d0-041d7f589c76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:25.299366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:25.423256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 677/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:14:25 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:29.476167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929122-476f-4be8-847e-8745729a98fd None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:29.477167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929122-476f-4be8-847e-8745729a98fd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:29.477677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929122-476f-4be8-847e-8745729a98fd None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:29.479635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 675/1352] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:14:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:29.485000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de81ea-74d3-4d77-bec3-109755ce39b8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:29.485878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de81ea-74d3-4d77-bec3-109755ce39b8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:29.486368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94de81ea-74d3-4d77-bec3-109755ce39b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:29.568288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-94de81ea-74d3-4d77-bec3-109755ce39b8 None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:29.569900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-94de81ea-74d3-4d77-bec3-109755ce39b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:29.643996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:29.681642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 678/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:14:29 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:14:31.103997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602831da-d368-42e8-9350-f0ad4f681550 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:31.104506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602831da-d368-42e8-9350-f0ad4f681550 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:31.104604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602831da-d368-42e8-9350-f0ad4f681550 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:31.105949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 676/1354] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:14:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:31.110924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2505c-9a20-4fd3-9a21-73169b8518fb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:31.111319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2505c-9a20-4fd3-9a21-73169b8518fb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:31.111667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2505c-9a20-4fd3-9a21-73169b8518fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:31.192037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-61a2505c-9a20-4fd3-9a21-73169b8518fb None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:31.193256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-61a2505c-9a20-4fd3-9a21-73169b8518fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:31.299028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 679/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:14:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:34.762498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09ae7-0de4-4c06-a65b-cc5a46019c9a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:34.763549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09ae7-0de4-4c06-a65b-cc5a46019c9a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:34.764880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda09ae7-0de4-4c06-a65b-cc5a46019c9a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:34.770706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 677/1356] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:34 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:34.772451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626658e-66c1-480f-ab22-6c0430b4b1bf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:34.772918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626658e-66c1-480f-ab22-6c0430b4b1bf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:34.773026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1626658e-66c1-480f-ab22-6c0430b4b1bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:34.819532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:34.859020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1626658e-66c1-480f-ab22-6c0430b4b1bf None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:34.860294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1626658e-66c1-480f-ab22-6c0430b4b1bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:34.963552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 680/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:35.020296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:35.021104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:35.021735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:35.024791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 678/1358] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 31 14:14:35 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:35.033826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e05ab-a746-431c-a986-158f1b339f56 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:35.034549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e05ab-a746-431c-a986-158f1b339f56 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:35.037370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c11e05ab-a746-431c-a986-158f1b339f56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:35.118634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c11e05ab-a746-431c-a986-158f1b339f56 None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:35.119896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c11e05ab-a746-431c-a986-158f1b339f56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:35.178923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:35.277255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 681/1359] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 31 14:14:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:35.285336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:35.286952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:35.446146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b7e9d-4bba-4902-808a-b705b6e6ede8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:35.523274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:35.567849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:35.568550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:35.568932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:35.569306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:35.571066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:35.665313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:35.667061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d05a0a-67f8-4b7a-81cd-1132d2e93cef service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:35.691715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 679/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:37.918284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:37.918814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:37.918814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:37.920299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 682/1361] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:37.926292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748407-65dd-4b05-9927-384899464ee8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:37.926762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748407-65dd-4b05-9927-384899464ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:37.926866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7748407-65dd-4b05-9927-384899464ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:38.014948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d7748407-65dd-4b05-9927-384899464ee8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:38.016174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d7748407-65dd-4b05-9927-384899464ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:38.057356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:38.130535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 680/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:38.180838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:38.184397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:38.353095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b285f2c0-9585-4aea-985d-d87e10387daf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:38.363872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:38.479633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:38.480269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:38.480551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:38.480829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:38.482475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:38.574098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:38.575735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35035252-1292-4263-8a00-95dbc1c4ab45 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:38.595135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 683/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:40.930031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745af56-d9e0-4842-9fea-bd1241b7c317 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:40.930571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745af56-d9e0-4842-9fea-bd1241b7c317 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:40.930571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1745af56-d9e0-4842-9fea-bd1241b7c317 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:40.931952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 681/1364] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:40.937564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787fe2d-2537-469a-8dfc-70bbebe83534 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:40.938027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787fe2d-2537-469a-8dfc-70bbebe83534 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:40.938108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8787fe2d-2537-469a-8dfc-70bbebe83534 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:40.954969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:41.025924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8787fe2d-2537-469a-8dfc-70bbebe83534 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:41.027158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8787fe2d-2537-469a-8dfc-70bbebe83534 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:41.136020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 684/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:41.182582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:41.182791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:41.183319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:41.185078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 682/1366] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 31 14:14:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:41.191367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64563c2-e7e2-4b49-8f4e-1b53e331e747 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:41.191689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64563c2-e7e2-4b49-8f4e-1b53e331e747 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:41.191969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64563c2-e7e2-4b49-8f4e-1b53e331e747 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:41.272013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b64563c2-e7e2-4b49-8f4e-1b53e331e747 None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:41.273229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b64563c2-e7e2-4b49-8f4e-1b53e331e747 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:41.310219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:41.406333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 685/1367] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 31 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:41.414738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:41.416346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:41.574185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb21bc6-e1aa-4b81-9b98-4a93dc28cc8f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:41.595031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:41.694823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:41.695466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:41.695752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:41.696027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:41.697681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:41.791935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:41.793484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdbd196d-7497-444c-9e69-60ff355ec086 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:41.813960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 683/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:42.845515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aefe-0448-466a-8ed5-344b3ed6ebca None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:42.846629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aefe-0448-466a-8ed5-344b3ed6ebca None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:42.847172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece7aefe-0448-466a-8ed5-344b3ed6ebca None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:42.849261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 686/1369] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:42.854413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46627-f5da-445a-b34c-6f3169e4fc27 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:42.855490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46627-f5da-445a-b34c-6f3169e4fc27 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:42.855992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa46627-f5da-445a-b34c-6f3169e4fc27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:42.894335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:42.968750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8fa46627-f5da-445a-b34c-6f3169e4fc27 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:42.970259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8fa46627-f5da-445a-b34c-6f3169e4fc27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:43.077950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 684/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:43.423590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:43.423590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:43.423590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:43.427372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 687/1371] 10.222.0.40 () {54 vars in 964 bytes} [Sun Jul 31 14:14:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:43.431461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa19a-0644-45ed-8929-97dcbd7fa1b6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:43.431461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa19a-0644-45ed-8929-97dcbd7fa1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:43.431461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356aa19a-0644-45ed-8929-97dcbd7fa1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:43.524753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-356aa19a-0644-45ed-8929-97dcbd7fa1b6 None None] MFA Rules not processed for user `5bd24147c8c44ff7a9800bc406d51953`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:43.526722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-356aa19a-0644-45ed-8929-97dcbd7fa1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:43.580737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:43.618221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 685/1372] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Jul 31 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:14:43.625933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:43.627605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:43.741459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e80f73-a73c-4557-bc6e-8c45b704356a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:43.783265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:43.819696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:43.820943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:43.821190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:43.821474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:43.823506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:43.881807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1554ba-76db-4da8-beda-429e659d0bbd None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:43.882324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1554ba-76db-4da8-beda-429e659d0bbd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:43.882423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1554ba-76db-4da8-beda-429e659d0bbd None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:43.884129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 686/1373] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:14:43.889325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5630a-edbb-475c-bac4-59610e7eabf8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:43.889604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5630a-edbb-475c-bac4-59610e7eabf8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:43.890037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e5630a-edbb-475c-bac4-59610e7eabf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:43.946384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:43.948275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:43.970302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c3e5630a-edbb-475c-bac4-59610e7eabf8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:43.971534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c3e5630a-edbb-475c-bac4-59610e7eabf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:43.974762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 688/1374] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:44.088691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 687/1375] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:14:44.140901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:44.142451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:44.223173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:44.258925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6661df6-76f9-419c-a328-a7eaaf27a01e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:44.377015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:44.377633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:44.377912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:44.378715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:44.380358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:44.457598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:44.468234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:44.469819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357ae087-e74f-4e01-a6e5-0c3e0857f880 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:44.487181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 689/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:46.575644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:46.576126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:46.576126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:46.577439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 688/1377] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:46.582886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1bd71-d27a-486a-9689-ae3c16159b19 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:46.583286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1bd71-d27a-486a-9689-ae3c16159b19 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:46.583472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf1bd71-d27a-486a-9689-ae3c16159b19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:46.649605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-abf1bd71-d27a-486a-9689-ae3c16159b19 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:46.651038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-abf1bd71-d27a-486a-9689-ae3c16159b19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:46.762056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 690/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:46.806508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:46.808067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:46.862138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:46.982179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af95c0ca-d2f7-4fee-9a01-2ead492751a6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:47.119664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:47.120171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:47.120446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:47.120713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:47.122342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:47.134690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:47.215066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:47.216681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5164cc7-2025-4731-b35b-2becbc1a8afd service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:47.234683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 689/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:49.293060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:49.293592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:49.293592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:49.294973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 691/1380] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:49.300856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d955b1-125f-49d7-9a94-7aae86db301f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:49.301293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d955b1-125f-49d7-9a94-7aae86db301f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:49.301491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d955b1-125f-49d7-9a94-7aae86db301f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:49.372734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a7d955b1-125f-49d7-9a94-7aae86db301f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:49.373949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a7d955b1-125f-49d7-9a94-7aae86db301f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:49.469924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:49.487315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 690/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:49.531949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:49.533442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:49.598610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:49.683978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6835dfa-a954-45bc-a10d-764583fef981 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:49.809068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:49.809685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:49.809959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:49.810224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:49.811958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:49.835733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:49.906585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:49.908202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01072ba7-55db-47f0-8374-6b66ce961ef5 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:49.927016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 692/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:51.996834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:51.997369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:51.997369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:51.998743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 691/1383] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:52.004247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2515f-a2d2-4031-aba7-168ab94f8a3d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:52.004658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2515f-a2d2-4031-aba7-168ab94f8a3d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:52.004802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd2515f-a2d2-4031-aba7-168ab94f8a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:52.077763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9cd2515f-a2d2-4031-aba7-168ab94f8a3d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:52.079443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9cd2515f-a2d2-4031-aba7-168ab94f8a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:52.190655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:52.195404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 693/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:14:52.239805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:52.241367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:52.390179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9641a68-114e-477a-a63c-257a14b78f64 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:52.466065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:52.521436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:52.522066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:52.522346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:52.522622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:52.524298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:52.615412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:52.617092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46753767-05bc-4cf8-8308-228f1cd3ac66 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:52.639038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 692/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:54.628138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:54.628682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:54.628682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:54.630041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 694/1386] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:54.635670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbd817-3bb7-4beb-b08f-94b9335d5c6d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:54.636133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbd817-3bb7-4beb-b08f-94b9335d5c6d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:54.636459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bbd817-3bb7-4beb-b08f-94b9335d5c6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:54.712154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-24bbd817-3bb7-4beb-b08f-94b9335d5c6d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:54.713375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-24bbd817-3bb7-4beb-b08f-94b9335d5c6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:54.740648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:54.811787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 693/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:14:54.858223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:54.859781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:55.046609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112a116-9d85-4b47-9fd2-3e54bad3d45d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:55.135944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:55.186151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:55.186793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:55.187067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:55.187337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:55.188994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:55.282995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:55.285133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff5e37d6-fc09-4d76-a976-99770f0d162f service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:55.303356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 695/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:57.338682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:57.339253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:57.339253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:57.340624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 694/1389] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:57.345867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b180b0-cad8-4bc5-aa60-cad711872089 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:57.346846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b180b0-cad8-4bc5-aa60-cad711872089 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:57.347147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b180b0-cad8-4bc5-aa60-cad711872089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:57.412031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f1b180b0-cad8-4bc5-aa60-cad711872089 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:57.413146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f1b180b0-cad8-4bc5-aa60-cad711872089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:57.447039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:57.509418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 696/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:57.553596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:57.555096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:57.698700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c45fa5-be4a-4c46-a096-c0e881f94597 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:57.708310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:57.811514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:57.812184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:57.812462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:57.812734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:57.814642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:57.902705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:57.904353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80bf6220-e5c8-4f89-b59f-00065d9a7c41 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:57.921855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 695/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:59.020975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceedd7e-b21f-421c-b11f-f55ee606239b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.021487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceedd7e-b21f-421c-b11f-f55ee606239b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.021487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceedd7e-b21f-421c-b11f-f55ee606239b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.022818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 697/1392] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:59.028061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e29644-a867-4c24-bcbe-6396b18d5081 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.028482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e29644-a867-4c24-bcbe-6396b18d5081 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.028608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e29644-a867-4c24-bcbe-6396b18d5081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.044995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:59.098851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-21e29644-a867-4c24-bcbe-6396b18d5081 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:59.100072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-21e29644-a867-4c24-bcbe-6396b18d5081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:59.197969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 696/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:59.252877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.253160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.253459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.254799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 698/1394] 10.222.0.40 () {54 vars in 966 bytes} [Sun Jul 31 14:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:59.259900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a79d4d-b676-4566-869d-61cc51cb5952 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.260191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a79d4d-b676-4566-869d-61cc51cb5952 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.260470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a79d4d-b676-4566-869d-61cc51cb5952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.336485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-97a79d4d-b676-4566-869d-61cc51cb5952 None None] MFA Rules not processed for user `42b59618e59540d1bcd5c2397b004422`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:14:59.337613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-97a79d4d-b676-4566-869d-61cc51cb5952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:59.372277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:59.415356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 697/1395] 10.222.0.40 () {58 vars in 1063 bytes} [Sun Jul 31 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:14:59.423108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:14:59.425183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:59.528864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200256e1-10dc-4b93-9862-21a37039cb75 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:14:59.592205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:14:59.604745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:14:59.605373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.605657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.605943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.607588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:14:59.718872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:14:59.720334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994f3f8e-a58c-4db0-bda8-ee17c0c9809a service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:14:59.739783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 699/1396] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:14:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:14:59.952520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.952801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.953075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:14:59.954495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 698/1397] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:14:59.960568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fc0a8-83d0-4fd1-8db5-a3d226f025b8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:14:59.960847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fc0a8-83d0-4fd1-8db5-a3d226f025b8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:14:59.961116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7fc0a8-83d0-4fd1-8db5-a3d226f025b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:00.036086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3d7fc0a8-83d0-4fd1-8db5-a3d226f025b8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:00.038042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3d7fc0a8-83d0-4fd1-8db5-a3d226f025b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.081273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:00.150910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 700/1398] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:00.196726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:00.198265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.353023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebae950-2bac-478f-8126-f49a8111df34 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:00.398022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:00.474312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:00.474976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:00.475206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:00.475480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:00.477085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.482707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184251f7-77f4-459e-a171-7338d5ed6aea None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:00.482997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184251f7-77f4-459e-a171-7338d5ed6aea None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:00.483276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184251f7-77f4-459e-a171-7338d5ed6aea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:00.550221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-184251f7-77f4-459e-a171-7338d5ed6aea None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:00.551525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-184251f7-77f4-459e-a171-7338d5ed6aea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.567044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:00.568737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12687c89-1012-4fda-b409-2602b4862538 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:00.587539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 699/1399] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:00.645248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 701/1400] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:15:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:15:00.696791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:00.697084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:00.697385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:00.698837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 700/1401] 10.222.0.40 () {54 vars in 943 bytes} [Sun Jul 31 14:15:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:00.706567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280a5d-e59c-4c3c-abc5-e157b0385d54 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:00.706855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280a5d-e59c-4c3c-abc5-e157b0385d54 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:00.707150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80280a5d-e59c-4c3c-abc5-e157b0385d54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:00.758451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:00.795223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-80280a5d-e59c-4c3c-abc5-e157b0385d54 None None] MFA Rules not processed for user `9d498354314244afa4445a1ed99d15cf`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:00.796546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80280a5d-e59c-4c3c-abc5-e157b0385d54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.908066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 702/1402] 10.222.0.40 () {58 vars in 1040 bytes} [Sun Jul 31 14:15:00 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:00.918418 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:00.920014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:00.973499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:01.072829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b4599-b3f5-4b6e-bc24-6fc02b22398f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:01.175466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:01.176027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:01.176303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:01.176572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:01.178228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:01.208560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:01.260998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:01.262586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e89f69-1ac2-488f-a762-624ebea33bf6 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:01.270790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 701/1403] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:15:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 14:15:02.684077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:02.684609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:02.684609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:02.685992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 703/1404] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:15:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:02.697445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ddd1c-ef56-4efb-a889-adf543f3083e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:02.697993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ddd1c-ef56-4efb-a889-adf543f3083e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:02.698643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ddd1c-ef56-4efb-a889-adf543f3083e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:02.791274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5f9ddd1c-ef56-4efb-a889-adf543f3083e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:02.793774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5f9ddd1c-ef56-4efb-a889-adf543f3083e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:02.966576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 702/1405] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 274 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:03.011248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:03.013086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:03.138098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:03.202952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad528a5-eca5-4103-b0cc-7af3b02a4d60 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:03.331156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:03.331841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:03.332132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:03.332410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:03.334085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:03.397095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:03.429877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:03.431542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45aae9c-b56a-4e61-bb55-b84d5d2fcf68 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:03.449942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 704/1406] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:05.508071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:05.508602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:05.508602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:05.510212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 703/1407] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:15:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:05.515105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f514f2-a688-410e-a951-afcc9d0cf36c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:05.515540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f514f2-a688-410e-a951-afcc9d0cf36c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:05.515699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f514f2-a688-410e-a951-afcc9d0cf36c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:05.589887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-40f514f2-a688-410e-a951-afcc9d0cf36c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:05.591202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-40f514f2-a688-410e-a951-afcc9d0cf36c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:05.693817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 705/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:05.738179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:05.739940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:05.766179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:05.907304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d269041-eaf1-4656-9ca7-cb48d5605cf6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:06.012697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:06.036736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:06.037288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:06.037565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:06.037835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:06.039539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:06.147028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:06.149099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36b692f-1a86-464e-811a-0409ef647e1d service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:06.169080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 704/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:15.692244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=85 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:16.166547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=52 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:23.756715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c66f-716b-47fa-9337-b45ac444464a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:23.757228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c66f-716b-47fa-9337-b45ac444464a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:23.757228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9431c66f-716b-47fa-9337-b45ac444464a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:23.758576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 706/1410] 10.222.0.40 () {54 vars in 918 bytes} [Sun Jul 31 14:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:23.763927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f430c-3da0-400c-9ff7-dfcb099a2a26 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:23.764323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f430c-3da0-400c-9ff7-dfcb099a2a26 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:23.764484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f430c-3da0-400c-9ff7-dfcb099a2a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:23.843897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-943f430c-3da0-400c-9ff7-dfcb099a2a26 None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:23.845108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-943f430c-3da0-400c-9ff7-dfcb099a2a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:23.948654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 705/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Jul 31 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:23.963945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1f54-392f-42c4-ac4a-87de23f8128c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:23.964234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1f54-392f-42c4-ac4a-87de23f8128c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:23.964512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-668a1f54-392f-42c4-ac4a-87de23f8128c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:23.965904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 707/1412] 10.222.0.40 () {54 vars in 918 bytes} [Sun Jul 31 14:15:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:23.970468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11931d32-9b83-4502-a2bd-affb4819cb0f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:23.970787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11931d32-9b83-4502-a2bd-affb4819cb0f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:23.971067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11931d32-9b83-4502-a2bd-affb4819cb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:24.041693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:24.044298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-11931d32-9b83-4502-a2bd-affb4819cb0f None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:24.046062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-11931d32-9b83-4502-a2bd-affb4819cb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:24.159270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 706/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Sun Jul 31 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:26.595842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:26.596426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:26.596426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:26.597773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:26.602994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df446-77d7-4f32-bb1d-055ccf73185a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:26.603461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df446-77d7-4f32-bb1d-055ccf73185a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:26.603579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75df446-77d7-4f32-bb1d-055ccf73185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:26.669481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e75df446-77d7-4f32-bb1d-055ccf73185a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:26.670688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e75df446-77d7-4f32-bb1d-055ccf73185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:26.725011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:26.764733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:26.808128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:26.809674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:26.953952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad803e1-5c31-48d4-9531-24174cb2f6b3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:27.027860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:27.067837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:27.068459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:27.068749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:27.069018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:27.070639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:27.157901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:27.159544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad09b6a-7d35-4189-827f-1b52c35d39d2 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:27.177873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 709/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:28.088750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:28.089312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:28.089312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:28.090682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:28.096158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7389e8-2171-4471-8ada-b72b75f1581a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:28.096598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7389e8-2171-4471-8ada-b72b75f1581a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:28.096696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7389e8-2171-4471-8ada-b72b75f1581a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:28.172662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9b7389e8-2171-4471-8ada-b72b75f1581a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:28.173896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9b7389e8-2171-4471-8ada-b72b75f1581a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:28.200002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:28.270061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:28.422921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:28.424461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:28.568267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c5f9c-b08f-4768-9e02-39aa45674398 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:28.607209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:28.679071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:28.679651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:28.679935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:28.680212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:28.681840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:28.768854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:28.770437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7c4e06-c684-4824-b07a-dc0ffd7b0f96 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:28.788057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 709/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:29.675014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:29.675616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:29.675616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:29.677114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:29.682003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79ea8b-3fc2-49a3-af8c-c5a690159763 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:29.682391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79ea8b-3fc2-49a3-af8c-c5a690159763 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:29.682551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb79ea8b-3fc2-49a3-af8c-c5a690159763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:29.737362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:29.758874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fb79ea8b-3fc2-49a3-af8c-c5a690159763 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:29.760130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fb79ea8b-3fc2-49a3-af8c-c5a690159763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:29.858574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:30.011060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:30.012509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:30.156182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041e0b52-e095-4989-b3ed-61e43dca3a02 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:30.168155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:30.268143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:30.268692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:30.268971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:30.269242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:30.270845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:30.356654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:30.358226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbe5a39-4f32-44f8-ab95-85b3c5c978f2 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:30.376752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 712/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:31.234037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:31.234589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:31.234589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:31.236015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:31.241036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16394-1b9d-4762-b960-47b2f1cd5068 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:31.241477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16394-1b9d-4762-b960-47b2f1cd5068 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:31.241581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d16394-1b9d-4762-b960-47b2f1cd5068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:31.264860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:31.321383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f1d16394-1b9d-4762-b960-47b2f1cd5068 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:31.322598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f1d16394-1b9d-4762-b960-47b2f1cd5068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:31.417735 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:31.567847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:31.569399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:31.694661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:31.729672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f511a09c-069f-4ede-90df-78606d04d214 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:31.839861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:31.840475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:31.840756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:31.841029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:31.842651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:31.916787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:31.930995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:31.932622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc5b912-be47-427a-82dc-af6a53896da6 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:31.951130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 712/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:32.842721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6673-3674-4616-84e8-0084f5436def None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:32.843274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6673-3674-4616-84e8-0084f5436def None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:32.843274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ec6673-3674-4616-84e8-0084f5436def None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:32.844693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:15:32.849420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e478936-a0fa-4996-b253-d30bcc1908ac None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:32.849850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e478936-a0fa-4996-b253-d30bcc1908ac None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:32.849970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e478936-a0fa-4996-b253-d30bcc1908ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:32.923593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3e478936-a0fa-4996-b253-d30bcc1908ac None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:32.924801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3e478936-a0fa-4996-b253-d30bcc1908ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:33.027093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:33.175388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6673-3674-4616-84e8-0084f5436def None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:33.176975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-24ec6673-3674-4616-84e8-0084f5436def None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:33.265698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:33.334746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ec6673-3674-4616-84e8-0084f5436def None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:33.456302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:33.456919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:33.457202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:33.457474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:33.459097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:33.507104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:33.548933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:33.550542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df809df0-752a-432b-984e-9b4a5f051053 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:33.568383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 715/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:34.409952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:34.410528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:34.410528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:34.411824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:34.416586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180026-e424-4089-993b-f84fed6b3acc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:34.417051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180026-e424-4089-993b-f84fed6b3acc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:34.417158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac180026-e424-4089-993b-f84fed6b3acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:34.492535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ac180026-e424-4089-993b-f84fed6b3acc None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:34.493745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ac180026-e424-4089-993b-f84fed6b3acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:34.588859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:34.736098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:34.737686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:34.785658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:34.882648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff96b923-1992-4137-9c7c-c5439b855a04 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:34.992818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:34.993371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:34.993649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:34.993921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:34.995598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:35.002953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:35.083270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:35.084884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2474a8-772d-4ddc-85f8-f3eb32401496 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:35.103172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 715/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:35.899114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:35.899752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:35.899752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:35.901160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:35.906146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ff3c-a749-4514-bd4d-3ba5038b3d94 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:35.906546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ff3c-a749-4514-bd4d-3ba5038b3d94 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:35.906701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce2ff3c-a749-4514-bd4d-3ba5038b3d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:35.980785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5ce2ff3c-a749-4514-bd4d-3ba5038b3d94 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:35.981989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5ce2ff3c-a749-4514-bd4d-3ba5038b3d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:36.047182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:36.076068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:36.225414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:36.226870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:36.244602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:36.374908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1c3408-e903-4c8e-8b41-0a86c4b92af2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:36.467070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:36.494103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:36.494748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:36.495028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:36.495299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:36.496967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:36.583707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:36.585316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647518ad-6e4b-4e87-9ecf-3e2a1ae18217 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:36.603269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 718/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:37.404751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:37.405279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:37.405279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:37.406580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:37.412678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dcec1-fbac-48cb-a1b8-9a92f57defeb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:37.413163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dcec1-fbac-48cb-a1b8-9a92f57defeb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:37.413452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7dcec1-fbac-48cb-a1b8-9a92f57defeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:37.490052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7a7dcec1-fbac-48cb-a1b8-9a92f57defeb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:37.491244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7a7dcec1-fbac-48cb-a1b8-9a92f57defeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:37.534198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:37.588860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:37.737733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:37.739245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:37.894238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d782ae3-9966-4577-9585-6e2105f8eab2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:37.947786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:38.009002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:38.009574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:38.009856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:38.010134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:38.011874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:38.099751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:38.101329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c58d42cf-5f5f-47c6-b51e-d944d9355cd6 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:38.119326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 718/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:38.931779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:38.932336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:38.932336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:38.933667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:15:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:38.938724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d02e4-8ad4-4efd-988c-58b02f5acaf1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:38.939400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d02e4-8ad4-4efd-988c-58b02f5acaf1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:38.939576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99d02e4-8ad4-4efd-988c-58b02f5acaf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:39.013763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f99d02e4-8ad4-4efd-988c-58b02f5acaf1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:39.014893 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f99d02e4-8ad4-4efd-988c-58b02f5acaf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:39.017019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:39.112175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:39.261360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:39.262876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:39.410441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c715a90-8eef-43f4-850b-6019bf3e931c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:39.436259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:39.522005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:39.522626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:39.522905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:39.523174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:39.524812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:39.612416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:39.614019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2deb12-efd8-49ed-9b0f-89748dc0caeb service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:39.632285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 721/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:41.548517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:41.549039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:41.549039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:41.550342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:41.555545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62171ad-c572-442a-b6d1-1a771fddd9d6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:41.555994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62171ad-c572-442a-b6d1-1a771fddd9d6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:41.556099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62171ad-c572-442a-b6d1-1a771fddd9d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:41.596864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:41.621718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c62171ad-c572-442a-b6d1-1a771fddd9d6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:41.622915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c62171ad-c572-442a-b6d1-1a771fddd9d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:41.718549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:41.756658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:41.758176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:41.862306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7258b8-c181-46ea-adcc-614d56b3f64a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:41.894238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:41.934164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:41.934778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:41.935056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:41.935334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:41.936987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:42.043048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:42.044514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e79e1c1-8d2d-4bff-8b12-79adccbca737 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:42.062403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 721/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:44.221571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243593-285f-4123-8419-db53fe184854 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:44.222114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243593-285f-4123-8419-db53fe184854 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:44.222114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63243593-285f-4123-8419-db53fe184854 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:44.223479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:44.228629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6baa-3634-4ee3-93a9-ae96b2339cbe None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:44.229067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6baa-3634-4ee3-93a9-ae96b2339cbe None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:44.229242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbe6baa-3634-4ee3-93a9-ae96b2339cbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:44.294573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fdbe6baa-3634-4ee3-93a9-ae96b2339cbe None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:44.295807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fdbe6baa-3634-4ee3-93a9-ae96b2339cbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:44.308891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:44.391992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:44.429980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243593-285f-4123-8419-db53fe184854 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:44.431595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-63243593-285f-4123-8419-db53fe184854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:44.533710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63243593-285f-4123-8419-db53fe184854 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:44.545337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:44.604973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:44.605582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:44.605859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:44.606130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:44.607756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:44.712722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:44.714153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaff216-4697-4529-8602-94f8cf588c4b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:44.732375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 724/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:47.132129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:47.132674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:47.132674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:47.134047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:47.139438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f6f8-5750-40a3-81ed-097f6e46b32e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:47.139861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f6f8-5750-40a3-81ed-097f6e46b32e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:47.140063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61f6f8-5750-40a3-81ed-097f6e46b32e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:47.186758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:47.205714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cb61f6f8-5750-40a3-81ed-097f6e46b32e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:47.206918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cb61f6f8-5750-40a3-81ed-097f6e46b32e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:47.301203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:47.340258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:47.341770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:47.444941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d9c71a-2dc6-4b51-9a3c-ab79f6dd5e52 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:47.482771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:47.516787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:47.517397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:47.517673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:47.517952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:47.519593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:47.630918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:47.632373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ad6737-0ceb-40b2-affe-6129651f8bce service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:47.650140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 724/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:49.847008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:49.847612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:49.847612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:49.848958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:49.854477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c975b-98d5-47f5-910e-d118ae968657 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:49.854945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c975b-98d5-47f5-910e-d118ae968657 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:49.855069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455c975b-98d5-47f5-910e-d118ae968657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:49.923616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-455c975b-98d5-47f5-910e-d118ae968657 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:49.924837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-455c975b-98d5-47f5-910e-d118ae968657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:49.946077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:50.020766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:50.059970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:50.061619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:50.165685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d3cb48-95ac-4d31-9b6d-61849827c000 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:50.184754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:50.359501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:50.360107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:50.360383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:50.360687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:50.362320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:50.468582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:50.469966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84082c4-3da5-4423-a396-6b1e30033f8b service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:50.488090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 727/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:52.607532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:52.608065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:52.608065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:52.609353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:52.615089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175793d5-5fb4-4c23-bcd4-2f0d05291759 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:52.615552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175793d5-5fb4-4c23-bcd4-2f0d05291759 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:52.615663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175793d5-5fb4-4c23-bcd4-2f0d05291759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:52.677386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:52.680610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-175793d5-5fb4-4c23-bcd4-2f0d05291759 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:52.682210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-175793d5-5fb4-4c23-bcd4-2f0d05291759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:52.775664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:52.812974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:52.814580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:52.917305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00daedb-edc7-4e14-aa09-efc5c18bc924 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:52.982709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:53.008315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:53.008934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:53.009211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:53.009485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:53.011130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:53.123040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:53.124499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65316f3a-297f-44f5-88ca-579b85879f97 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:53.142340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 727/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:55.249546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:55.250321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:55.250321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:55.251582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:55.256996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f272-b9fd-490d-91ec-52af1af53ff9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:55.257436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f272-b9fd-490d-91ec-52af1af53ff9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:55.257544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac69f272-b9fd-490d-91ec-52af1af53ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:55.322151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ac69f272-b9fd-490d-91ec-52af1af53ff9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:55.323383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ac69f272-b9fd-490d-91ec-52af1af53ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:55.354526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:55.422265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:55.461225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:55.462747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:55.562930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25269e0-dd25-46cb-950b-e5ff0193e917 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:55.592602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:55.637655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:55.638278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:55.638559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:55.638856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:55.640519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:55.749004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:55.750371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73453f2a-3661-4122-a4d9-d4b76cdf15fa service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:55.768876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 730/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:15:57.777945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:57.778492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:57.778994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:57.779929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:15:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:15:57.785884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317d1a-8901-4264-bf74-7af08155e0d3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:57.786297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317d1a-8901-4264-bf74-7af08155e0d3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:57.786438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7317d1a-8901-4264-bf74-7af08155e0d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:57.852995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c7317d1a-8901-4264-bf74-7af08155e0d3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:15:57.854213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c7317d1a-8901-4264-bf74-7af08155e0d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:57.862301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:57.951292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:15:57.988012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:15:57.989537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:58.091677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f5ea6-f94c-4807-8d8a-ecebb73d5d23 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:15:58.146792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:15:58.165059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:15:58.165678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:15:58.165959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:15:58.166232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:15:58.167897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:15:58.274735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:15:58.276192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad2e539-5ee2-4f6f-8d3e-210392c0dbd5 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:15:58.294770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 730/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:00.336949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:00.337586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:00.337809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:00.339195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:16:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:00.346213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a672-062c-4ec0-baf6-48b732296c6c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:00.346680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a672-062c-4ec0-baf6-48b732296c6c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:00.346777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2281a672-062c-4ec0-baf6-48b732296c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:00.414216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-2281a672-062c-4ec0-baf6-48b732296c6c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:00.415346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2281a672-062c-4ec0-baf6-48b732296c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:00.452462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:00.509689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:00.547822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:00.549406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:00.650316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234bf093-de8e-49a4-8e25-cbb27a3b73c5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:00.686598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:00.722776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:00.723397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:00.723705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:00.723979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:00.725589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:00.832183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:00.833610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e54b6f-3860-46e7-8629-203a0f2a8ece service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:00.851242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 733/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:02.891474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:02.892007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:02.892099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:02.893403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:16:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:02.898892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20084-a975-4656-abbf-d5783da42f14 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:02.899312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20084-a975-4656-abbf-d5783da42f14 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:02.899478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a20084-a975-4656-abbf-d5783da42f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:02.964621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e5a20084-a975-4656-abbf-d5783da42f14 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:02.965759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e5a20084-a975-4656-abbf-d5783da42f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:02.984615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:03.066346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:03.103206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:03.104762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:03.208739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a58b51-e2ee-4fe2-bd18-0f9d84063603 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:03.273929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:03.280958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:03.281576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:03.281853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:03.282121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:03.284366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:03.395943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:03.397415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24018332-4149-4737-9446-d32bd92eaa00 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:03.420000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 733/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:05.450018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:05.450613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:05.450613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:05.451996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:16:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:05.457293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43732f0f-c1ae-4db4-a165-01c90f0bc82f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:05.457774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43732f0f-c1ae-4db4-a165-01c90f0bc82f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:05.457884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43732f0f-c1ae-4db4-a165-01c90f0bc82f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:05.521468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-43732f0f-c1ae-4db4-a165-01c90f0bc82f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:05.522654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-43732f0f-c1ae-4db4-a165-01c90f0bc82f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:05.569297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:05.616479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:05.653297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:05.654823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:05.756901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64db8a2c-657a-44ab-81b5-2bd568a2e464 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:05.803709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:05.830336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:05.830953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:05.831231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:05.831535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:05.833149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:05.940291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:05.941987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7580bc61-53f0-44dc-a0a9-a7a211de7ef3 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:05.959873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 736/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:15.614695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=22 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:15.957565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=66 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:25.597030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92ef9-1aa8-4da7-b72f-2e0792358c97 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:25.597560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92ef9-1aa8-4da7-b72f-2e0792358c97 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:25.597560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be92ef9-1aa8-4da7-b72f-2e0792358c97 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:25.598860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 735/1471] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:25.604214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472071-2dc5-4399-b5fb-65f9e3501cbd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:25.604650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472071-2dc5-4399-b5fb-65f9e3501cbd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:25.604757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4472071-2dc5-4399-b5fb-65f9e3501cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:25.667978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f4472071-2dc5-4399-b5fb-65f9e3501cbd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:25.669163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f4472071-2dc5-4399-b5fb-65f9e3501cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:25.764107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 737/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:25.846573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:25.846857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:25.847187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:25.848537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 736/1473] 10.222.0.40 () {54 vars in 966 bytes} [Sun Jul 31 14:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:25.853451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eed5f-debe-4546-ba7e-d68da655208a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:25.853741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eed5f-debe-4546-ba7e-d68da655208a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:25.854041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61eed5f-debe-4546-ba7e-d68da655208a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:25.928131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b61eed5f-debe-4546-ba7e-d68da655208a None None] MFA Rules not processed for user `42b59618e59540d1bcd5c2397b004422`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:25.929332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b61eed5f-debe-4546-ba7e-d68da655208a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:25.949958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:26.004821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 738/1474] 10.222.0.40 () {58 vars in 1063 bytes} [Sun Jul 31 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:26.012366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:26.013855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:26.114376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ec3065-31cb-4c4b-a88f-1b6bf91d9d91 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:26.186138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:26.186761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:26.187042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:26.187328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:26.189014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:26.241649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:26.295786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:26.297216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10191b53-2433-4d3b-b7c8-1d1150ee1a50 service glance] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:26.315849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 737/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:28.386932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:28.387515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:28.387638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:28.388926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 739/1476] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:28.394839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27fa5-3c53-48f6-b67c-c31a2ed5d71a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:28.395273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27fa5-3c53-48f6-b67c-c31a2ed5d71a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:28.395374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a27fa5-3c53-48f6-b67c-c31a2ed5d71a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:28.461426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-75a27fa5-3c53-48f6-b67c-c31a2ed5d71a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:28.462625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-75a27fa5-3c53-48f6-b67c-c31a2ed5d71a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:28.555480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 738/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:28.578764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:28.580941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:28.703882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3656d0-e703-484f-bf22-adcca491b159 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:28.751199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:28.794770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:28.795378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:28.795684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:28.795953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:28.796640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:28.798358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:16:28.798358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:16:28.798358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:16:28.800027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244525ea-7618-46db-a749-41b5965c0a5d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:28.819025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 740/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Sun Jul 31 14:16:28 2022] GET /identity/v3/projects => generated 2553 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:16:30.857167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:30.857693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:30.857693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:30.859108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 739/1479] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:30.864888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517996c-f492-41c1-90fa-15bc5f0bb78d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:30.865311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517996c-f492-41c1-90fa-15bc5f0bb78d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:30.865442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6517996c-f492-41c1-90fa-15bc5f0bb78d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:30.932837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6517996c-f492-41c1-90fa-15bc5f0bb78d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:30.934044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6517996c-f492-41c1-90fa-15bc5f0bb78d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:31.028194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 741/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:31.071503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:31.073053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:31.083309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:31.216964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a1a8b9-a166-4740-9d48-0cabb8d6d0b8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:31.293885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:31.331479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:31.332091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:31.332366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:31.332634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:31.334239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:31.419583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:31.421157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6363af4d-5672-4dcc-a26e-61f6d6ce25af service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:31.438455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 740/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:33.616001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:33.616530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:33.616530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:33.617830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 742/1482] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:33.623132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10d96-37b5-4c27-945b-5ab2ee9d39fd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:33.623625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10d96-37b5-4c27-945b-5ab2ee9d39fd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:33.623739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d10d96-37b5-4c27-945b-5ab2ee9d39fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:33.688797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-99d10d96-37b5-4c27-945b-5ab2ee9d39fd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:33.689990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-99d10d96-37b5-4c27-945b-5ab2ee9d39fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:33.735582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:33.784695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 741/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:33.821442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:33.822963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:33.846962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:33.924157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd379dbd-b3f4-4a29-9585-c5203081b772 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:33.994709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:33.995464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:33.995756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:33.996029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:33.997689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:34.070566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:34.105027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:34.106512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2f2c0-b04e-4cdd-ab56-ae26e81aebc6 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:34.123994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 743/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:36.190000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:36.190535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:36.190535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:36.192171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 742/1485] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:36.197236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942399b-9712-4b5d-9e77-11db90bd8d2d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:36.197688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942399b-9712-4b5d-9e77-11db90bd8d2d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:36.197827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7942399b-9712-4b5d-9e77-11db90bd8d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:36.264549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7942399b-9712-4b5d-9e77-11db90bd8d2d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:36.265770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7942399b-9712-4b5d-9e77-11db90bd8d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:36.358906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 744/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:36.397330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:36.398764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:36.498505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35e6720-18e5-4424-aad6-c3c7f21a354e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:36.568198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:36.569982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:36.570589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:36.571110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:36.571409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:36.573415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:36.682677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:36.684178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4758d063-38d3-48af-90b6-f26760a97284 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:36.702665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 743/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:38.886812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:38.887362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:38.887362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:38.888728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 745/1488] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:38.894392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be2ba-c2d7-46cc-9c53-c65973b5d6e1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:38.894823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be2ba-c2d7-46cc-9c53-c65973b5d6e1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:38.894940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b35be2ba-c2d7-46cc-9c53-c65973b5d6e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:38.960372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b35be2ba-c2d7-46cc-9c53-c65973b5d6e1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:38.961806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b35be2ba-c2d7-46cc-9c53-c65973b5d6e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:39.013536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:39.062255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 744/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:16:39.102774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:39.104319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:39.139965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:39.210065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e81d3-0018-4b7d-819f-e1f1323fef34 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:39.282431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:39.283008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:39.283291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:39.283603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:39.285217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:39.363880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:39.393051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:39.394489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ec2a63-b8f3-470a-98aa-5d4aecd8636f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:39.413523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 746/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:41.639059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:41.639635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:41.639635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:41.641005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 745/1491] 10.222.0.40 () {54 vars in 926 bytes} [Sun Jul 31 14:16:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:16:41.646708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94c4f-5e3e-4932-a86f-d6821ec81ad7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:41.647129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94c4f-5e3e-4932-a86f-d6821ec81ad7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:41.647238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f94c4f-5e3e-4932-a86f-d6821ec81ad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:41.712233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f0f94c4f-5e3e-4932-a86f-d6821ec81ad7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:16:41.713434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0f94c4f-5e3e-4932-a86f-d6821ec81ad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:41.809963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 747/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Sun Jul 31 14:16:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:16:41.850455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:16:41.852031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:41.954364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b678ea3-46be-4977-9d4d-d74e38881d0a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:16:42.027289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:16:42.027941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:16:42.028223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:16:42.028503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:16:42.030139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:16:42.039118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:42.137943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:16:42.139381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-650cbcdc-a114-4101-a0fd-174443a57b35 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:16:42.160862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 746/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:16:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:16:51.808181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=82 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:16:52.156717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=50 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:37.637599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:37.638205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:37.638205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:37.711710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:18:37.712895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:37.806708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 748/1494] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:18:37.815430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:37.815925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:37.816035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:37.892789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:18:37.893942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:37.994348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 747/1495] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:18:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:18:37.998891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.000168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:38.012240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a435814-0ce9-43f2-b3ff-af780e881ffd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.072826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:38.107008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.107665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.107950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.108226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.108920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:18:38.109231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.110232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.110232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:18:38.110232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:18:38.111121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.125576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 749/1496] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:18:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:18:38.133868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.143195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efbe808-71fc-4ee3-afd4-e8f0621dfdfb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.203711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:38.237557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.238175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.238455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.238727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.239402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.241152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.241152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:18:38.241152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:18:38.242335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.296156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 748/1497] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:18:38 2022] POST /identity/v3/projects => generated 379 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:18:38.300995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.302569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:38.336218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543b6e25-baeb-48dd-87b8-8e840309b980 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.429248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.429781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.430060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.430332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.430983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.432028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.432028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:18:38.432028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:18:38.433230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.482852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 750/1498] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:18:38 2022] POST /identity/v3/users => generated 290 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:18:38.487211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.488786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:38.550803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8299f90f-f47f-42d9-a215-ca98188331b1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.631723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:38.643638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.644258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.644538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.644812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.645513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.646521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.646521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:18:38.646521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:18:38.647405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.660099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 749/1499] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:18:38.664273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.665824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:38.677247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd1f36d-46b2-4a45-942e-39056d94936d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.701872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:38.772561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.773179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.773457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.773754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.774444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.775464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.775464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:18:38.775464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:18:38.776354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4f8351-4188-4fce-9670-30a5b436a2e6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.791323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 751/1500] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:18:38 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:18:38.795554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:38.797166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:38.837451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48739a4e-a829-40a2-bc3b-b8be2ed05a30 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:38.932813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:38.933480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.933757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.934027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] PATH_INFO: `/v3/projects/7b417c063be84d7983b924dbc4c08796/users/2e5e6bbdc6f449dd92e9a69bfef4c26c/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:38.956885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b417c063be84d7983b924dbc4c08796, user_id=2e5e6bbdc6f449dd92e9a69bfef4c26c, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:38.957906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:38.957906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:18:38.957906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:18:38.959907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4567752f-7810-4b91-81f7-88c3bf5ba8c1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:38.984623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 750/1501] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:18:38 2022] PUT /identity/v3/projects/7b417c063be84d7983b924dbc4c08796/users/2e5e6bbdc6f449dd92e9a69bfef4c26c/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:18:38.996680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daa966-e460-433c-b1e2-d042670d1a87 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:38.996964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daa966-e460-433c-b1e2-d042670d1a87 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:38.997246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74daa966-e460-433c-b1e2-d042670d1a87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.052718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-74daa966-e460-433c-b1e2-d042670d1a87 None None] MFA Rules not processed for user `2e5e6bbdc6f449dd92e9a69bfef4c26c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:18:39.053909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-74daa966-e460-433c-b1e2-d042670d1a87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:39.130770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 752/1502] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:18:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:18:39.144550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:39.144840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:39.145109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.146504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 751/1503] 10.222.0.40 () {54 vars in 964 bytes} [Sun Jul 31 14:18:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:18:39.152247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90ccf-2b60-4129-8638-fc8b987b56bc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:39.152532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90ccf-2b60-4129-8638-fc8b987b56bc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:39.152809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f90ccf-2b60-4129-8638-fc8b987b56bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.197351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:39.248372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-96f90ccf-2b60-4129-8638-fc8b987b56bc None None] MFA Rules not processed for user `5bd24147c8c44ff7a9800bc406d51953`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:18:39.249491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96f90ccf-2b60-4129-8638-fc8b987b56bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:39.323552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 753/1504] 10.222.0.40 () {58 vars in 1061 bytes} [Sun Jul 31 14:18:39 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:18:39.331525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:39.333017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:39.381573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:39.435483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30aa35d-5dfe-413d-a5b1-c8c8fa7eaed8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:39.511817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:39.512362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:39.512642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:39.512926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.514564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:39.613337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:39.614763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:39.618256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:39.633286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 752/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:18:39 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:18:39.678439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912906-4c71-47f0-b675-5da49dea09fc None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:39.678866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912906-4c71-47f0-b675-5da49dea09fc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:39.678984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2912906-4c71-47f0-b675-5da49dea09fc None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.680442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 754/1506] 10.222.0.40 () {52 vars in 830 bytes} [Sun Jul 31 14:18:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:18:39.683974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:39.685169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:39.781904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca3f023-6c72-4959-8893-ca79d60383d1 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:39.876194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:39.876894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:39.877102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:39.877417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] PATH_INFO: `/v3/users/2e5e6bbdc6f449dd92e9a69bfef4c26c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:39.889362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e5e6bbdc6f449dd92e9a69bfef4c26c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:39.890374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:39.890374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:18:39.890374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:18:39.891601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:39.901605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:39.961445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e5e6bbdc6f449dd92e9a69bfef4c26c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:39.984306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e5e6bbdc6f449dd92e9a69bfef4c26c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:39.984751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e5e6bbdc6f449dd92e9a69bfef4c26c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:39.985267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e5e6bbdc6f449dd92e9a69bfef4c26c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:39.998746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e5e6bbdc6f449dd92e9a69bfef4c26c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:40.013281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 753/1507] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:18:39 2022] DELETE /identity/v3/users/2e5e6bbdc6f449dd92e9a69bfef4c26c => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:18:40.018310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:40.018604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:40.018885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:40.074160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:18:40.075279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:40.155036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:40.169094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 755/1508] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:18:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:18:40.178590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:40.179841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:40.281586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d89ea8ce-f42a-440b-ba68-c8d8bd88cb1a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:40.342875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:40.365637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:40.366284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:40.366569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:40.366846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:40.368646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:40.478754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:40.480251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88d6ad4-6475-411d-98e5-e04dc53e123a service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:40.498205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 754/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:18:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:18:40.839475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:18:40.840880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:18:40.852545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a42a69-6a75-4bdf-9c73-a87ea5493de9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:18:40.949212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:18:40.949860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:18:40.950137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:18:40.950405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] PATH_INFO: `/v3/projects/7b417c063be84d7983b924dbc4c08796` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:18:40.960316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b417c063be84d7983b924dbc4c08796)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:18:40.961336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:18:40.961336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:18:40.961336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:18:40.962532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:18:41.018323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b417c063be84d7983b924dbc4c08796', 'request_id': 'req-a2e11149-5788-432a-9cea-d38795ed7307'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:41.028338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b417c063be84d7983b924dbc4c08796', 'request_id': 'req-a2e11149-5788-432a-9cea-d38795ed7307'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:41.042271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b417c063be84d7983b924dbc4c08796', 'request_id': 'req-a2e11149-5788-432a-9cea-d38795ed7307'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:41.066762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] The token cache is being invalidate because project 7b417c063be84d7983b924dbc4c08796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:18:41.067052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:41.067507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a2e11149-5788-432a-9cea-d38795ed7307 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:18:41.068917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 756/1510] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:18:40 2022] DELETE /identity/v3/projects/7b417c063be84d7983b924dbc4c08796 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:18:50.496024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=62 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:18:51.064703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:20:40.821799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:40.822396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:40.822396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:40.823996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:40.828946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd096-fc30-4287-9178-8820c5405b62 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:40.829387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd096-fc30-4287-9178-8820c5405b62 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:40.829492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631bd096-fc30-4287-9178-8820c5405b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:40.912806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-631bd096-fc30-4287-9178-8820c5405b62 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:40.914037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-631bd096-fc30-4287-9178-8820c5405b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:41.016136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:20:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:41.168789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:41.170903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:41.288537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb0eb73-6138-4521-9dbf-e68d706cea9a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:41.294024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:41.294631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:41.294910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:41.295184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:41.303017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:41.304615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e34f32ef-5ece-427e-a1ab-67be176083cf service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:41.313670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 756/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:20:42.152110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b635e7-904e-46fb-b000-e1408e5c7ccd None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:42.152549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b635e7-904e-46fb-b000-e1408e5c7ccd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:42.152652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b635e7-904e-46fb-b000-e1408e5c7ccd None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:42.154110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 758/1514] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:20:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:42.159015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127e-3e37-4f54-aae3-7d02023dd142 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:42.159468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127e-3e37-4f54-aae3-7d02023dd142 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:42.159578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2a127e-3e37-4f54-aae3-7d02023dd142 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:42.179663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7f2a127e-3e37-4f54-aae3-7d02023dd142 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:42.180857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7f2a127e-3e37-4f54-aae3-7d02023dd142 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:42.188563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 757/1515] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:42.338013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b635e7-904e-46fb-b000-e1408e5c7ccd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:42.353413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b635e7-904e-46fb-b000-e1408e5c7ccd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:42.358779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:42.359414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:42.359842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:42.360123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:42.369827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:42.371410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640e4c82-c9d2-4553-b7df-4760b6db7b5f service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:42.381146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 759/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:20:42.558167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ef90-53e6-4fb6-adde-cbfc13b8a55c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:42.558451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ef90-53e6-4fb6-adde-cbfc13b8a55c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:42.558728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7312ef90-53e6-4fb6-adde-cbfc13b8a55c None None] PATH_INFO: `/v3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:42.560102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 758/1517] 10.222.0.17 () {54 vars in 925 bytes} [Sun Jul 31 14:20:42 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:20:42.566363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f1bf-a431-47bc-8964-add05eab249f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:42.566653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f1bf-a431-47bc-8964-add05eab249f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:42.566933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a16f1bf-a431-47bc-8964-add05eab249f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:42.625746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1a16f1bf-a431-47bc-8964-add05eab249f None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:42.626956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1a16f1bf-a431-47bc-8964-add05eab249f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:42.643818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:20:42.710362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 760/1518] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Jul 31 14:20:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:51.558965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b132a7-d7b7-425e-bc83-5a53e44a3286 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:51.559507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b132a7-d7b7-425e-bc83-5a53e44a3286 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:51.559507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b132a7-d7b7-425e-bc83-5a53e44a3286 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:51.560806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 759/1519] 10.222.0.40 () {54 vars in 961 bytes} [Sun Jul 31 14:20:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:51.565708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f232a-4fb1-4735-9b30-eee6a68b822b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:51.566155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f232a-4fb1-4735-9b30-eee6a68b822b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:51.566254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5f232a-4fb1-4735-9b30-eee6a68b822b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:51.586177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3a5f232a-4fb1-4735-9b30-eee6a68b822b None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:51.587445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3a5f232a-4fb1-4735-9b30-eee6a68b822b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:51.595978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 761/1520] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Jul 31 14:20:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:51.603225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b132a7-d7b7-425e-bc83-5a53e44a3286 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:51.617062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b132a7-d7b7-425e-bc83-5a53e44a3286 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:51.621950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:51.622555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:51.622836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:51.623112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:51.631894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:51.633466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cd661e-5a79-48e6-aaf6-d520c5e30749 service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:51.642924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 760/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:51 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:20:53.632517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72f98-97b6-468a-92b6-bf1e5ab93af8 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:53.633065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72f98-97b6-468a-92b6-bf1e5ab93af8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:53.633065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b72f98-97b6-468a-92b6-bf1e5ab93af8 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:53.634425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:20:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:53.639063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a9cd-8fba-4301-bdf8-5c20e5ee11c8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:53.639525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a9cd-8fba-4301-bdf8-5c20e5ee11c8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:53.639634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1238a9cd-8fba-4301-bdf8-5c20e5ee11c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:53.659280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-1238a9cd-8fba-4301-bdf8-5c20e5ee11c8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:53.660531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1238a9cd-8fba-4301-bdf8-5c20e5ee11c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:53.668941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:20:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:53.822313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b72f98-97b6-468a-92b6-bf1e5ab93af8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:53.837125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b72f98-97b6-468a-92b6-bf1e5ab93af8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:53.842416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:53.843016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:53.843293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:53.843596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:53.852211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:53.853768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8f2fc5-6980-4469-b8d5-ed108b2ccd1f service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:53.862943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 763/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:20:54.690136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99001-7c8c-4414-ad95-e5654b14d035 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:54.690706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99001-7c8c-4414-ad95-e5654b14d035 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:54.690706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde99001-7c8c-4414-ad95-e5654b14d035 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:54.692115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Sun Jul 31 14:20:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:54.697441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d942-4483-4787-ad02-90fd5fb6e14f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:54.697871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d942-4483-4787-ad02-90fd5fb6e14f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:54.697969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb25d942-4483-4787-ad02-90fd5fb6e14f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:54.717102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bb25d942-4483-4787-ad02-90fd5fb6e14f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:54.718283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bb25d942-4483-4787-ad02-90fd5fb6e14f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:54.726024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Sun Jul 31 14:20:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:54.879531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde99001-7c8c-4414-ad95-e5654b14d035 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:54.893522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde99001-7c8c-4414-ad95-e5654b14d035 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:54.898884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:54.899515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:54.899795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:54.900064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:54.907927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:54.909490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c09a566-ed1d-4b38-a681-394d889b43a6 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:54.919103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 763/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:20:58.809410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:58.809978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:58.809978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:58.811340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:20:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:20:58.816776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929c1c-d243-46ce-a6d5-aa3c6b6b826b None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:58.817190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929c1c-d243-46ce-a6d5-aa3c6b6b826b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:58.817368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45929c1c-d243-46ce-a6d5-aa3c6b6b826b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:58.836492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-45929c1c-d243-46ce-a6d5-aa3c6b6b826b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:20:58.837722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-45929c1c-d243-46ce-a6d5-aa3c6b6b826b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:58.845509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:20:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:20:58.919221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:20:58.920965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:20:58.998772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6baabfd2-63c0-4dbf-a722-f5515492043f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:20:59.002301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:20:59.002909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:20:59.003196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:20:59.003499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:20:59.012203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:20:59.013620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06bc8949-70d4-47fa-b702-a55f33cf4745 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:20:59.023261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 766/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:20:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:21:00.997866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fab494-b8c9-47ec-9d78-13ea96775989 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:00.998445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fab494-b8c9-47ec-9d78-13ea96775989 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:00.998445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fab494-b8c9-47ec-9d78-13ea96775989 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:00.999773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:21:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:21:01.004941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837928-4bc6-4a1d-ad40-800f7e4d3a65 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:01.005331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837928-4bc6-4a1d-ad40-800f7e4d3a65 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:01.005487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1837928-4bc6-4a1d-ad40-800f7e4d3a65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:01.024551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e1837928-4bc6-4a1d-ad40-800f7e4d3a65 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:21:01.025745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e1837928-4bc6-4a1d-ad40-800f7e4d3a65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:21:01.033515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:21:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:21:01.069227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fab494-b8c9-47ec-9d78-13ea96775989 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:21:01.079946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fab494-b8c9-47ec-9d78-13ea96775989 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:21:01.083404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:21:01.084078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:01.084337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:01.084610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:01.093239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:21:01.094655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ad5206-ff05-438a-af38-8d1575833974 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:21:01.104323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 766/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:21:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:21:04.548644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:04.549217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:04.549217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:04.550523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:21:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:21:04.556156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91cac-2289-4ca2-83b0-55e73a5db173 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:04.556578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91cac-2289-4ca2-83b0-55e73a5db173 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:04.556709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b91cac-2289-4ca2-83b0-55e73a5db173 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:04.575948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e4b91cac-2289-4ca2-83b0-55e73a5db173 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:21:04.577145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e4b91cac-2289-4ca2-83b0-55e73a5db173 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:21:04.584900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:21:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:21:04.886770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:21:04.888471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:21:04.951148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:21:04.964599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5d545a-ceaf-4de9-b128-c53e3971fa3a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:21:04.968143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:21:04.968750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:04.969028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:04.969296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:04.978541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:21:04.980000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03df46a-dd3b-411c-989f-250d8b981d57 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:21:04.989093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 769/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:21:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:21:06.997806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:06.998334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:06.998334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:06.999764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Sun Jul 31 14:21:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:21:07.004960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952135a-21a1-4b53-8a37-5f720bcb6c20 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:07.005395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952135a-21a1-4b53-8a37-5f720bcb6c20 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:07.005516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9952135a-21a1-4b53-8a37-5f720bcb6c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:07.024503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9952135a-21a1-4b53-8a37-5f720bcb6c20 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:21:07.025693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9952135a-21a1-4b53-8a37-5f720bcb6c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:21:07.033485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Sun Jul 31 14:21:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:21:07.333677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:21:07.335907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:21:07.348539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c58b9a7-4e22-4ef7-80f0-41f60689f2b7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:21:07.352611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:21:07.353211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:21:07.353490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:21:07.353761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:21:07.362383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:21:07.363839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50203aac-c705-47c4-9706-0567dbdbf0d9 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:21:07.373652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 769/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:21:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:21:17.019780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=11 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:21:17.369461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:34.302946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.304129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.304129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.328698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:34.331053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:34.343923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 771/1540] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:34.353688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.354801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.355322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.386885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:34.388775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:34.404130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 770/1541] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:34.409841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.425880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f1678e-3962-4bc6-9550-bb69b27a36f4 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.431303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.432002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.432286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.432560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.433319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:34.433626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.435457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.452894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 772/1542] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:34.461678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.472136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddcd503-a668-45d6-b427-d2976178d0eb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.483938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.484974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.485489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.486008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.486945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.489416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.559355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 771/1543] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/projects => generated 373 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:22:34.563589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.580248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbe47a8-f783-45ff-a2a3-d29b0242bff9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.652155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.653232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.653790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.654352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.655246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.657432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.702299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 773/1544] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/users => generated 287 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:34.706373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.721858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eff32f1-8b52-4f3f-b74e-86fe2fbd6e4f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.731794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.732436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.732721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.732998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.733705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.735510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.752659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 772/1545] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:34.757290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.769313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5b9525-0e79-4659-8100-89a66e90570b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.779982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.780916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.781427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.781931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.782863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.784755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5365946c-2960-49fa-928a-3fc1cffdf784 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.800988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 774/1546] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:34 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:34.805833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:34.821851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78aaa20-72ea-4bde-811e-c5929fb92367 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:34.831461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:34.832282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.832576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.832854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] PATH_INFO: `/v3/projects/e3e2f3b3c56746e396e504040e2fae2e/users/680fc6372f5f400e8b092fab936e34f3/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.853920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3e2f3b3c56746e396e504040e2fae2e, user_id=680fc6372f5f400e8b092fab936e34f3, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:34.857085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a8b203-6619-4e22-945b-e018ec2f6430 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:34.891017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 773/1547] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:34 2022] PUT /identity/v3/projects/e3e2f3b3c56746e396e504040e2fae2e/users/680fc6372f5f400e8b092fab936e34f3/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:34.891927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75b1c-b559-4806-aac1-38f6421ab788 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.892454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75b1c-b559-4806-aac1-38f6421ab788 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.892952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd75b1c-b559-4806-aac1-38f6421ab788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.909031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eb76-da15-4e4d-b733-d43f606476b9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:34.911226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eb76-da15-4e4d-b733-d43f606476b9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:34.911226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0464eb76-da15-4e4d-b733-d43f606476b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:34.932185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8bd75b1c-b559-4806-aac1-38f6421ab788 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:34.933411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8bd75b1c-b559-4806-aac1-38f6421ab788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:34.972392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0464eb76-da15-4e4d-b733-d43f606476b9 None None] MFA Rules not processed for user `680fc6372f5f400e8b092fab936e34f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:34.973614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0464eb76-da15-4e4d-b733-d43f606476b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.030847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 775/1548] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:22:35.032417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883254-bf8d-4ca3-8773-263c2319023a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.032707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883254-bf8d-4ca3-8773-263c2319023a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.032979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f883254-bf8d-4ca3-8773-263c2319023a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.052415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0f883254-bf8d-4ca3-8773-263c2319023a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.054125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0f883254-bf8d-4ca3-8773-263c2319023a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.060493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 774/1549] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.061773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa12cc-e75d-4342-b093-1ed5a3a33442 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.062054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa12cc-e75d-4342-b093-1ed5a3a33442 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.062173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 776/1550] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.062600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0aa12cc-e75d-4342-b093-1ed5a3a33442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.063445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcd47-67cd-4c87-b5ed-e88f8a733a9c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.063835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcd47-67cd-4c87-b5ed-e88f8a733a9c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.063966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efcd47-67cd-4c87-b5ed-e88f8a733a9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.083859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e0aa12cc-e75d-4342-b093-1ed5a3a33442 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.085094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e0aa12cc-e75d-4342-b093-1ed5a3a33442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.087247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-77efcd47-67cd-4c87-b5ed-e88f8a733a9c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.088490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-77efcd47-67cd-4c87-b5ed-e88f8a733a9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.096722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 777/1551] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.103445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a7e8-7310-44b9-96c1-16d3e951edb0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.103445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a7e8-7310-44b9-96c1-16d3e951edb0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.103445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf99a7e8-7310-44b9-96c1-16d3e951edb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.103792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 775/1552] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.103792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.103792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.103792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.119675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cf99a7e8-7310-44b9-96c1-16d3e951edb0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.120872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cf99a7e8-7310-44b9-96c1-16d3e951edb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.123448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.124991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.129291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 778/1553] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.130527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84360-b393-48c3-95a8-e0fee265845d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.130850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84360-b393-48c3-95a8-e0fee265845d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.131080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84360-b393-48c3-95a8-e0fee265845d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.134355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 776/1554] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.135037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.144701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1f9952-5b26-4fc1-b565-d071e7176825 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.150095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.150736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.151013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.151280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.152050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.153976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aa179c8-a930-4c9a-af82-da397e62d286 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.158789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-12d84360-b393-48c3-95a8-e0fee265845d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.160083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-12d84360-b393-48c3-95a8-e0fee265845d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.169379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 779/1555] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.170580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a79c-97e5-4d16-a9d5-0ea5da73139b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.170865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a79c-97e5-4d16-a9d5-0ea5da73139b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.171142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0519a79c-97e5-4d16-a9d5-0ea5da73139b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.187101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 777/1556] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 371 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.188726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dead6e-4d20-4760-b8d2-6dca96e9ad06 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.189014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dead6e-4d20-4760-b8d2-6dca96e9ad06 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.189292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dead6e-4d20-4760-b8d2-6dca96e9ad06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.197626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0519a79c-97e5-4d16-a9d5-0ea5da73139b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.198843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0519a79c-97e5-4d16-a9d5-0ea5da73139b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.209121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-73dead6e-4d20-4760-b8d2-6dca96e9ad06 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.210320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-73dead6e-4d20-4760-b8d2-6dca96e9ad06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.228189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.249508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.259412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 780/1557] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.260768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071c7b-436e-4449-882c-9be71bb2bfc2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.261053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071c7b-436e-4449-882c-9be71bb2bfc2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.261328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d071c7b-436e-4449-882c-9be71bb2bfc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.275168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 778/1558] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.276713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfd74b-bb3f-4f5e-95d9-aef3919981d6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.277614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfd74b-bb3f-4f5e-95d9-aef3919981d6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.277899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdfd74b-bb3f-4f5e-95d9-aef3919981d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.283713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1d071c7b-436e-4449-882c-9be71bb2bfc2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.285573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1d071c7b-436e-4449-882c-9be71bb2bfc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.295819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 781/1559] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.299892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.299892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.299892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.300261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0bdfd74b-bb3f-4f5e-95d9-aef3919981d6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.301444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0bdfd74b-bb3f-4f5e-95d9-aef3919981d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.310114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 779/1560] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.311315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.311631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.311905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.317488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.318796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.327372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 782/1561] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.328162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.331532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.332735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.338011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76be15e-5d20-4b08-91db-6c79673b2cca None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.341740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 780/1562] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.342487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.344351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.345221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.345501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.345772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.346462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.346763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.348413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.352442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77891136-b817-49d0-a5a9-b2c0fbed97d9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.357208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.357817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.358078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.358351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.359015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.360179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 783/1563] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.360321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:35.360321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:22:35.360321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:22:35.360867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.361247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cdf1330-363f-43ec-a7aa-60fb9e8e7a1f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.372548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7732da2a-63bb-4485-a276-88160ba30126 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.377686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.378476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.378759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.379033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.379759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.380065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.382324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.383148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 781/1564] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/users => generated 292 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.387255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.387575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.387859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.402146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 784/1565] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.402146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.405445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35a190d-14d4-41b8-be7e-c15846514a79 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.410253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.410821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.411093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.411364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.412084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.412387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.414544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.414802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:35.416088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:35.424437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 782/1566] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:35.425203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.433339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 785/1567] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.433339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.435033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee503f5-d2c8-4c98-9f9f-49db4b064e53 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.440002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.440611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.440890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.441165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.441866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.442168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.442606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18170455-3b99-4e53-9ed6-766f7ac165c4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.443239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:35.443239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:22:35.443239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:22:35.444132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.447297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.447942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.448223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.448496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.449179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.449853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.451452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.456510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 783/1568] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.457314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.464574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 786/1569] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.465451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.468267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34353031-ec56-468a-a657-81cd338d5898 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.473193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.473769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.474044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.474318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.475523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.475877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.477052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d475d1b3-52bb-445b-9005-cdd3a6f7bc29 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.477448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.481633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.482229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.482511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.482785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.483476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.484657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:35.484657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:22:35.484657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:22:35.485843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.489292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 784/1570] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.490273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.502002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887da51c-e98b-40e8-8a21-998ddfba7ec9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.506706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.507281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.507584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.507849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.508538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.510103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.517085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 787/1571] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.517085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.523239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 785/1572] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:35.523239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.532580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d17fb8f-64c4-43c3-97f9-0bcf9a589856 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.579706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5598488-9bfd-425e-a770-f5f882595fa5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.585110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.585755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.586055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.586348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.587034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.588505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.588986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.589147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.589405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.589682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.590383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:35.590691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.592299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.605379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 786/1573] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:35.606543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.616081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1340d153-2314-4dc7-b76d-9a07b9ee7ca3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.620885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 788/1574] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 383 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.621695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.654741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.672945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.673495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.673770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.674041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.674693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.676686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.681820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.684230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf326a9-3fc2-4be1-bc05-74ef41916009 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.689471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.690215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.690499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.690775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.691501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.693490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.704002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 787/1575] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.704958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.729436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 789/1576] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.729436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.770666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66860712-bded-4d07-a719-cd71e1455849 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.786551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f75f7c0-23ba-4557-98fe-c4622454d2c3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.791889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.792499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.792782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.793057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.793751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.795675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.821593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 790/1577] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 375 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.822493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.829339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.829970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.830249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.830522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.831187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.833188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.866037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 788/1578] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 377 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.866037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.885627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-add25c7e-2720-4f80-943e-8fa10f496af4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.923067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ae3b5d-3615-4708-977e-2f8a05b6815a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:35.927526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.928213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.928493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.928764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] PATH_INFO: `/v3/projects/423c9434d74b4c21aca451318e71c8f9/users/b98d92a2c77d4f278b36b0d783b2c6e7/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.929978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.942976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:35.943638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:35.944429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:35.944568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=423c9434d74b4c21aca451318e71c8f9, user_id=b98d92a2c77d4f278b36b0d783b2c6e7, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.944700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:35.945362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:35.947255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.947371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:35.962613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 789/1579] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:35 2022] PUT /identity/v3/projects/423c9434d74b4c21aca451318e71c8f9/users/b98d92a2c77d4f278b36b0d783b2c6e7/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:35.963249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.967619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:35.975197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 791/1580] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/users => generated 287 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:35.975197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:35.984833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c789dd48-1dd2-42e7-86c3-d1b5dc8f6841 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.028339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6a26e1-2cfb-4ec0-86b7-6f7d37029e8d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.033413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.034036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.034319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.034594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.035296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.037235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.041224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.041842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.042117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.042419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.043106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.045051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.064550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 790/1581] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/projects => generated 393 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.068573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 792/1582] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:35 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.068573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.069309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.076787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2df4d64-cdf2-45ba-8ca9-71b3e949e9e3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.132188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ecb714-e76d-4fc8-9a87-c6024175b8fd admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.134425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.135046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.135321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.135625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.136315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.137012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.137632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.137916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.138191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.138294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.138856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.140804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.162465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 791/1583] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/users => generated 283 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.163952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 793/1584] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/users => generated 295 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.164092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.164177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.174027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e071d2b7-c5b0-4ed5-8273-5c083f3ddef1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.174191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26468070-b2dc-4a70-b033-508135b317b4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.179191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.179327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.179879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.180036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.180169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.180298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.180435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.180565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.181099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.181214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.183022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.183133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.207017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 794/1585] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.207017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.207315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 792/1586] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/users => generated 288 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.207315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.217665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6e79a7-79c6-49ce-b201-2c0b7da061a8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.217808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb8c528-3cb8-4889-88f3-e9ad8f05142a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.222145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.222303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.222774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.222880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.223053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.223164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.223327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.223438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.224058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.224161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.226131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.226309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.244743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 793/1587] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.244743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.245158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 795/1588] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.245158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.254741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf379ee0-0475-45da-af62-eb4a3aaca198 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.255647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2045e863-e982-4203-a06c-e77c672d21d0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.259387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.260015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.260296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.260569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.261029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.261237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.261899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.262318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.262706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.262870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.263641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.265816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.272904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 796/1589] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.274699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.286171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c62d4b-3f16-494b-9e72-d5b4acb72944 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.289183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 794/1590] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/users => generated 297 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:36.290058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.294270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.294883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.295170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.295465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.296162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.297771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.299643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a848bb-ce0e-4cea-891e-f44307fc11d7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.304664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.305245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.305520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.305790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.306469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.308146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.308831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 797/1591] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.312572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.318638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 795/1592] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.319587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.322513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad36d386-c718-456e-95c9-839f774e3529 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.327557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.328161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.328444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.328721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.329039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8e0c5f-23c4-4d77-9ae4-bb8dc4cfa5cf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.329411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.331142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.332506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=12 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:36.333857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.334984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.335263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.335575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.336253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.337802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.345351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 798/1593] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.346584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.351515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 796/1594] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.352606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.356889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c934f06b-d639-4e58-94a4-a882134c2454 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.361490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.362083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.362366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.362639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765543a-43a4-4b4b-984b-338505775467 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.363076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.363470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.365061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.367185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.367919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.368203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.368477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] PATH_INFO: `/v3/projects/423c9434d74b4c21aca451318e71c8f9/users/b98d92a2c77d4f278b36b0d783b2c6e7/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.370410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=423c9434d74b4c21aca451318e71c8f9, user_id=b98d92a2c77d4f278b36b0d783b2c6e7, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.373088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.378509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 799/1595] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.378509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.378744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:INSERT=11 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:36.385608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 797/1596] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/423c9434d74b4c21aca451318e71c8f9/users/b98d92a2c77d4f278b36b0d783b2c6e7/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.386482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.387819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e4831c-0a27-4b1b-bc1a-ac8b1ed75629 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.444246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.444765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.445052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.445326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.446015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.447635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.448286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189b49a6-22ad-4eec-bad9-2a1fac885425 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.453544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.454158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.454440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.454825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.455461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.457055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.457923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 800/1597] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.458728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.468032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 798/1598] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.468777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.468889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf8c676e-cd23-4298-b6f6-d19a3afa603f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.473329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.473925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.474209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.474488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.475787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.477379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.479347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4fb500-be77-4992-8294-8c12c36cd41b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.484105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.484699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.484981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.485253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.485932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.487538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.488074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 801/1599] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.489045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.497370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 799/1600] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.498171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.498671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08772e1-fec6-4c46-9695-3f6edbaff847 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.503497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.504094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.504373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.504648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.505333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.506903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.507520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cfb258-09ed-447d-8efe-2a00a991960b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.512023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.512614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.512901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.513177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.513857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.515465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72adb4bb-4760-4772-911f-a2a2fba78e7f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.516696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 802/1601] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.517457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.527010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be3a26c-020f-4251-bd30-ed2f610f2b7b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.528982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 800/1602] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.530255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.530541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.530813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.532579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.533314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.533680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.533964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] PATH_INFO: `/v3/projects/eb1fb9777eab40bab0ae6d87404179aa/users/c5a5bac128e648c180843db576447937/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.551480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eb1fb9777eab40bab0ae6d87404179aa, user_id=c5a5bac128e648c180843db576447937, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.551848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:36.551848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:22:36.551848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:22:36.553876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.558394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] MFA Rules not processed for user `b98d92a2c77d4f278b36b0d783b2c6e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:36.559675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:36.566864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 803/1603] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/eb1fb9777eab40bab0ae6d87404179aa/users/c5a5bac128e648c180843db576447937/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.568282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.628734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b204c38e-e8d9-4db3-b420-7bec0a0f3894 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.633311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.633977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.634255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.634530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] PATH_INFO: `/v3/projects/9097fd66951e457ba46812ff17db7501/users/d6e8399080a942f1b689fc8c6b42c717/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.639123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 801/1604] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:36.639990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.651005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5986bbf6-2a2f-4a86-a71c-4c84b6f4f57d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.652212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9097fd66951e457ba46812ff17db7501, user_id=d6e8399080a942f1b689fc8c6b42c717, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.654869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.655842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.656459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.656742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.657087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.657701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.659381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.669666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 804/1605] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/9097fd66951e457ba46812ff17db7501/users/d6e8399080a942f1b689fc8c6b42c717/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.673482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 802/1606] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:36.673617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.673716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.682319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e2c4e7-7dc5-491a-b591-aa114588dbcc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.691352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:36.725594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:36.738909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70354364-98ba-4d77-9614-e6ff374fc42c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.744157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.744835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.745138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.745306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.745428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] PATH_INFO: `/v3/projects/47a8451d52c245b0b6fe3effe4a582cf/users/1aaa0f40d1814f33845446336402aafe/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.746007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.746285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.746556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] PATH_INFO: `/v3/projects/ae629f9940d446f2ab39e9a49b3f4753/users/53dab2ec3f2e4973893fb838c3d1d545/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.761710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae629f9940d446f2ab39e9a49b3f4753, user_id=53dab2ec3f2e4973893fb838c3d1d545, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.763095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a8451d52c245b0b6fe3effe4a582cf, user_id=1aaa0f40d1814f33845446336402aafe, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.764412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.766012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.779541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 803/1607] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/ae629f9940d446f2ab39e9a49b3f4753/users/53dab2ec3f2e4973893fb838c3d1d545/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.780363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.780622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 805/1608] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/47a8451d52c245b0b6fe3effe4a582cf/users/1aaa0f40d1814f33845446336402aafe/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.781421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.843973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41fa919a-d982-42cf-8669-0ffff16bfabb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.844212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1b4b5f-a95b-4a89-b2f4-9950f83ae118 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.848474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.848626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.849136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.849236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.849412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.849514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.849681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] PATH_INFO: `/v3/projects/af02bce9aa3c48a99a340d8f94f20a1d/users/6012a1e24c0443de8b50b6b1f9d3acdf/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.849786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] PATH_INFO: `/v3/projects/f4ef4d34a8114051a67c51e01cb2b8eb/users/db0c001088aa4c86b4562c369deb6c46/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.864611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af02bce9aa3c48a99a340d8f94f20a1d, user_id=6012a1e24c0443de8b50b6b1f9d3acdf, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.864840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4ef4d34a8114051a67c51e01cb2b8eb, user_id=db0c001088aa4c86b4562c369deb6c46, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:36.867367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.867536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2451a7b-fc44-48fe-bf40-7a61708d2c65 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:36.884841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 804/1609] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/f4ef4d34a8114051a67c51e01cb2b8eb/users/db0c001088aa4c86b4562c369deb6c46/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.885582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 806/1610] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/af02bce9aa3c48a99a340d8f94f20a1d/users/6012a1e24c0443de8b50b6b1f9d3acdf/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:36.886357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.896306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.896874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.897732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:36.924836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] MFA Rules not processed for user `c5a5bac128e648c180843db576447937`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:36.926048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:36.944508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1863910e-82a1-43e1-9c03-b0cb31a1fc59 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:36.982452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:36.992957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 805/1611] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:36 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:36.994613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:36.998658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:36.999293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:36.999721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:36.999997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.048515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:37.067224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.068744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e046de1-e327-42f8-9015-ca22215f0e84 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.070754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cfa5eb3-5d7c-4bb5-954f-8c6a8ffb9eef None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.075934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.076676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.077049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.077390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] PATH_INFO: `/v3/projects/832629dd8594494ab41b6596b2d73076/users/10a4469c46f64feb819f3b19a0f4e89b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.079385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 807/1612] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:37.082462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47693e-36e9-4e03-8521-a504728f15b4 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.082753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47693e-36e9-4e03-8521-a504728f15b4 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.083036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff47693e-36e9-4e03-8521-a504728f15b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.092986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=832629dd8594494ab41b6596b2d73076, user_id=10a4469c46f64feb819f3b19a0f4e89b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.095681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa7c6fc-6b8a-4fcc-bded-83f10f4d05eb admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.102403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ff47693e-36e9-4e03-8521-a504728f15b4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.103732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ff47693e-36e9-4e03-8521-a504728f15b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.111144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 806/1613] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:36 2022] PUT /identity/v3/projects/832629dd8594494ab41b6596b2d73076/users/10a4469c46f64feb819f3b19a0f4e89b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:37.112612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3de-a0cd-47ae-96c8-4bb06eeabdce None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.113329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3de-a0cd-47ae-96c8-4bb06eeabdce None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.113669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbd3de-a0cd-47ae-96c8-4bb06eeabdce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.138425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cfcbd3de-a0cd-47ae-96c8-4bb06eeabdce None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.139666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cfcbd3de-a0cd-47ae-96c8-4bb06eeabdce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.177884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 808/1614] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.179958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc354-3c53-4cb1-864f-18b2a42b3481 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.180246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc354-3c53-4cb1-864f-18b2a42b3481 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.180518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304dc354-3c53-4cb1-864f-18b2a42b3481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.211159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 807/1615] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.217510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fae76-0c4e-4553-b045-a4baa8ed840e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.217864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fae76-0c4e-4553-b045-a4baa8ed840e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.218220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62fae76-0c4e-4553-b045-a4baa8ed840e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.219153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-304dc354-3c53-4cb1-864f-18b2a42b3481 None None] MFA Rules not processed for user `1aaa0f40d1814f33845446336402aafe`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.220479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-304dc354-3c53-4cb1-864f-18b2a42b3481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.248341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d62fae76-0c4e-4553-b045-a4baa8ed840e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.249604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d62fae76-0c4e-4553-b045-a4baa8ed840e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.258772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 808/1616] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.260244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.260536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.260815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.293313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] MFA Rules not processed for user `6012a1e24c0443de8b50b6b1f9d3acdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.295564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.303710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 809/1617] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.306394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8dbb-bf22-4459-97fb-8ac27829e72a None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.306848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8dbb-bf22-4459-97fb-8ac27829e72a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.307277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93b8dbb-bf22-4459-97fb-8ac27829e72a None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.308849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 810/1618] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 31 14:22:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:37.315068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.315773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.316056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.339917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.342098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.355586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 811/1619] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.355586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.369826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:37.379289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 809/1620] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.380086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.432671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafb61e0-f16f-45ef-904c-1f78917d9483 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.436345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.437026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.437314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.437585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.447055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.448631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.449729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:37.455496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0fe0184-0f56-4c60-ab03-dada94f99441 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.460873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.461629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.462465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.463073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.463746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 812/1621] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:37.472730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.474614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.477310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.488396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793fee5f-4217-4678-bfd8-995ce1b2000a service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.493393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.494025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.494653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.494816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.504551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 810/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:37.505564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.507206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e77206-14ca-4578-9fd5-c22bdda1892d service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.508292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.524152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 813/1623] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:37.525764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4543251-9ddf-4f7a-84d5-1c9b81713b6e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.526077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4543251-9ddf-4f7a-84d5-1c9b81713b6e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.526403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4543251-9ddf-4f7a-84d5-1c9b81713b6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.527686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2a9318-098a-4314-9c67-468e7e5285bf service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.534822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.535809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.536233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.536606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.538153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.540714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.577015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e4543251-9ddf-4f7a-84d5-1c9b81713b6e None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.578378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e4543251-9ddf-4f7a-84d5-1c9b81713b6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.589462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 811/1624] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/projects => generated 363 bytes in 84 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:37.590547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.665201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3f0c2c-3a34-4c3d-9886-17c758c25cbf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.667714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 814/1625] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.669374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90e047-5346-4e3d-838f-10eef740b042 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.669505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.670016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90e047-5346-4e3d-838f-10eef740b042 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.670147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.670318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.670726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.670995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90e047-5346-4e3d-838f-10eef740b042 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.674065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 815/1626] 10.222.0.40 () {54 vars in 962 bytes} [Sun Jul 31 14:22:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:37.675098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90e047-5346-4e3d-838f-10eef740b042 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.747849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.749284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.758433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee90e047-5346-4e3d-838f-10eef740b042 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.761650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 812/1627] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:37.762906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.764344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.765139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.765588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.766031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.766857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.769050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad836292-0b7a-403d-b742-2efeb05a03bf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.772779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac710f9-b247-43e1-9552-5c10498f41bd service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:37.778129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:37.778791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.779069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.779340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] PATH_INFO: `/v3/users/680fc6372f5f400e8b092fab936e34f3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.782519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] RBAC: Authorizing `identity:delete_user(user_id=680fc6372f5f400e8b092fab936e34f3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:37.784500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:37.794407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 816/1628] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:37.795919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:37.796164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:37.796452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:37.817675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:37.819019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.825717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:37.849462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '680fc6372f5f400e8b092fab936e34f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:37.868604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '680fc6372f5f400e8b092fab936e34f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:37.869692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '680fc6372f5f400e8b092fab936e34f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:37.870609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '680fc6372f5f400e8b092fab936e34f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:37.880042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:INSERT=1,keystone:DELETE=9 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:37.883301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '680fc6372f5f400e8b092fab936e34f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:37.904329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 813/1629] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:37 2022] DELETE /identity/v3/users/680fc6372f5f400e8b092fab936e34f3 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:37.907354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.907354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.913211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 817/1630] 10.222.0.40 () {58 vars in 1059 bytes} [Sun Jul 31 14:22:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:37.914381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:37.924942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:37.941663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d487f1-a0d1-4728-8a37-177db201a1bd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.005817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30955570-c4ca-4d9b-92b6-55d636adbe92 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.010427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.011134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.011445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.011843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.012441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.012698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.013210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.013622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.014403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7998445-a0b9-4534-877e-1c25c1d7fb57 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.014499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.017226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.027099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 818/1631] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:38.028600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.028898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.029189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.053983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:38.055173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.068901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 819/1632] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:38.069346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.084100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff9d9f-6f46-4c56-a830-0f326fd31fdf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.090609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.091338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.091625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.091873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.093832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.106139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.108836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.120117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 814/1633] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:38.137770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.162031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69aae55-55cf-42d8-8726-3069559d1675 service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.169323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.170648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.171001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.171451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] PATH_INFO: `/v3/projects/931265c8c88f415683a5689bad4e237e/users/b7fa7cf30f7b4a62b90936015db276e1/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.185332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.188331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.205297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 820/1634] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:38 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:38.213734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] RBAC: Authorizing `identity:create_grant(project_id=931265c8c88f415683a5689bad4e237e, user_id=b7fa7cf30f7b4a62b90936015db276e1, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.216533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.218192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.221049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.241992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 815/1635] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:38 2022] PUT /identity/v3/projects/931265c8c88f415683a5689bad4e237e/users/b7fa7cf30f7b4a62b90936015db276e1/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:38.247558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.284428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:38.319961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d419791b-b21d-4b37-9172-783e215f202c service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.419580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.420718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.421633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.422370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.424807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474aefff-d5c9-4d39-852b-ca09c223097a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.435748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.436404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.436692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.436998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.439544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.442615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.443773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.447458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:38.448121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9218ef66-c367-43e2-b733-07d3c0c04648 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.465750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 816/1636] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:38 2022] GET /identity/v3/roles => generated 1420 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:38.480967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 821/1637] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:38.482832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.529892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b26f61-32e4-498a-9aa1-2cf38d776c37 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.542242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.542242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.542483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.542757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] PATH_INFO: `/v3/projects/931265c8c88f415683a5689bad4e237e/users/b7fa7cf30f7b4a62b90936015db276e1/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.548429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] RBAC: Authorizing `identity:create_grant(project_id=931265c8c88f415683a5689bad4e237e, user_id=b7fa7cf30f7b4a62b90936015db276e1, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.558062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db1f-a3d8-4ac1-8242-f5344c7ab689 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.558278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb0537-d8d1-4f26-b0ff-53ef9dff126f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.558381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db1f-a3d8-4ac1-8242-f5344c7ab689 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.558687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c326db1f-a3d8-4ac1-8242-f5344c7ab689 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.560450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 822/1638] 10.222.0.40 () {54 vars in 912 bytes} [Sun Jul 31 14:22:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:38.566288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf47a-8069-473e-9ee1-9e98d6a73954 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.566470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf47a-8069-473e-9ee1-9e98d6a73954 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.566746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bf47a-8069-473e-9ee1-9e98d6a73954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.588947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 817/1639] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:38 2022] PUT /identity/v3/projects/931265c8c88f415683a5689bad4e237e/users/b7fa7cf30f7b4a62b90936015db276e1/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:38.591300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960d2a-264e-4e59-a614-b57fc1f7974c None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.591300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960d2a-264e-4e59-a614-b57fc1f7974c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.591300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e960d2a-264e-4e59-a614-b57fc1f7974c None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.592888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 818/1640] 10.222.0.40 () {54 vars in 912 bytes} [Sun Jul 31 14:22:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:38.593891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.594210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.594462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.595801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-838bf47a-8069-473e-9ee1-9e98d6a73954 None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:38.597440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-838bf47a-8069-473e-9ee1-9e98d6a73954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.642141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] MFA Rules not processed for user `b7fa7cf30f7b4a62b90936015db276e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:38.643520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.757141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 823/1641] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:38.760045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560395-58af-4442-baae-4c2a50794b73 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.760493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560395-58af-4442-baae-4c2a50794b73 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.761293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25560395-58af-4442-baae-4c2a50794b73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.782710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 819/1642] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:38.786123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.790008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.809220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-25560395-58af-4442-baae-4c2a50794b73 None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:38.812061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-25560395-58af-4442-baae-4c2a50794b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.828755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 824/1643] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:22:38 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:38.829867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560395-58af-4442-baae-4c2a50794b73 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:38.831489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-25560395-58af-4442-baae-4c2a50794b73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:38.933972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f54769-0632-404a-b088-6ddfde910408 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.951775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.952518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.952806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.953097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.965874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25560395-58af-4442-baae-4c2a50794b73 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:38.971904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:38.972905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:38.973219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:38.973473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:38.981914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:38.983415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:38.991136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:39.002884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 820/1644] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:39.027152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:39.066471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 825/1645] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:38 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 237 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:39.159691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:39.169940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74664d5-f3e0-455f-9afa-b3cdbb4fe534 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:39.174298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:39.175308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:39.175890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:39.176630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:39.194291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:39.196188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:39.223025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 821/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:39.537280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:39.539373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:39.593329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:39.710925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0040d3d3-b129-4a78-848e-a245617bd46d service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:39.724470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:39.725931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:39.726217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:39.726499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] PATH_INFO: `/v3/users/6012a1e24c0443de8b50b6b1f9d3acdf` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:39.732334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6012a1e24c0443de8b50b6b1f9d3acdf)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:39.742485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:39.744137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:39.753594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: WARNING py.warnings [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:39.753594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: warnings.warn(msg) Jul 31 14:22:39.753594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: Jul 31 14:22:39.755464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:39.768224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6d59c4-3b9c-4e53-9f74-fe54ac6f8f70 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:39.780999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:39.781378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:39.781667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:39.782144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] PATH_INFO: `/v3/projects/e3e2f3b3c56746e396e504040e2fae2e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:39.784716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3e2f3b3c56746e396e504040e2fae2e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:39.785805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: WARNING py.warnings [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 31 14:22:39.785805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: warnings.warn(msg) Jul 31 14:22:39.785805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: Jul 31 14:22:39.789209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:39.917307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3e2f3b3c56746e396e504040e2fae2e', 'request_id': 'req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:39.948791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3e2f3b3c56746e396e504040e2fae2e', 'request_id': 'req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:39.951235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:39.973163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6012a1e24c0443de8b50b6b1f9d3acdf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:39.977873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3e2f3b3c56746e396e504040e2fae2e', 'request_id': 'req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.004376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] The token cache is being invalidate because project e3e2f3b3c56746e396e504040e2fae2e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:40.004376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.005007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.008073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 822/1647] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:39 2022] DELETE /identity/v3/projects/e3e2f3b3c56746e396e504040e2fae2e => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:40.012023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:40.015792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.042423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6012a1e24c0443de8b50b6b1f9d3acdf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.043448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6012a1e24c0443de8b50b6b1f9d3acdf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.079144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6012a1e24c0443de8b50b6b1f9d3acdf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.079946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b652573f-80c2-46b4-9615-245be39f5ea2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6012a1e24c0443de8b50b6b1f9d3acdf'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.151878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3cbd32-c63d-4d4f-b767-cf227efa2ada admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:40.152951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 826/1648] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:39 2022] DELETE /identity/v3/users/6012a1e24c0443de8b50b6b1f9d3acdf => generated 0 bytes in 617 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:40.159566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:40.160490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:40.160766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:40.247833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:40.249883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.280501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:40.282072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:40.282607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:40.283173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] PATH_INFO: `/v3/users/b98d92a2c77d4f278b36b0d783b2c6e7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:40.285393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b98d92a2c77d4f278b36b0d783b2c6e7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:40.292377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:40.415836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b98d92a2c77d4f278b36b0d783b2c6e7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.442512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 827/1649] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 286 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:40.447662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:40.447921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b98d92a2c77d4f278b36b0d783b2c6e7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.449180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.449970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b98d92a2c77d4f278b36b0d783b2c6e7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.453355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b98d92a2c77d4f278b36b0d783b2c6e7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.470276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b98d92a2c77d4f278b36b0d783b2c6e7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.476299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ae1825-039d-4ede-82ac-0f364ae0c69d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:40.490630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:40.491494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:40.491874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:40.492170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] PATH_INFO: `/v3/users/1aaa0f40d1814f33845446336402aafe` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:40.493775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1aaa0f40d1814f33845446336402aafe)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:40.499477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:40.500596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 823/1650] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:40 2022] DELETE /identity/v3/users/b98d92a2c77d4f278b36b0d783b2c6e7 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:40.501570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:40.508560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.535720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:40.587363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:40.605828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1aaa0f40d1814f33845446336402aafe'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.633027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1aaa0f40d1814f33845446336402aafe'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.637113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1aaa0f40d1814f33845446336402aafe'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.641136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7905b92-d063-4fc2-a30a-de254216bbc3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:40.655637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:40.656568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:40.657066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:40.657441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:40.661936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1aaa0f40d1814f33845446336402aafe'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.662911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a351ad94-f8bc-4c80-a161-e8813b453074 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1aaa0f40d1814f33845446336402aafe'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:40.663292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.696688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 828/1651] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:40 2022] DELETE /identity/v3/users/1aaa0f40d1814f33845446336402aafe => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:40.702438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:40.704477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:40.705072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:40.742790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:40.744360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:40.810174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:40.812466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:40.831173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 824/1652] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:40.892859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 829/1653] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:40.943348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:40.964592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:41.022636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b35f59-7782-4235-ba4a-fef6f17bcb57 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:41.147704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:41.150828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:41.151219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:41.153197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:41.180612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:41.182807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9576a7c-f95b-4c22-8f0c-9c01a033d097 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:41.206317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 825/1654] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 265 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:41.700884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:41.704432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:41.743009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf6eeec-c8ff-40d2-a431-9b5ff7ab30d2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:41.777710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:41.781032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:41.781442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:41.784005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] PATH_INFO: `/v3/projects/af02bce9aa3c48a99a340d8f94f20a1d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:41.786669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] RBAC: Authorizing `identity:delete_project(project_id=af02bce9aa3c48a99a340d8f94f20a1d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:41.800747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:41.954781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af02bce9aa3c48a99a340d8f94f20a1d', 'request_id': 'req-54c0f02c-276b-44af-a86e-c758d1cd3410'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:41.972157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af02bce9aa3c48a99a340d8f94f20a1d', 'request_id': 'req-54c0f02c-276b-44af-a86e-c758d1cd3410'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:41.990474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af02bce9aa3c48a99a340d8f94f20a1d', 'request_id': 'req-54c0f02c-276b-44af-a86e-c758d1cd3410'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:42.012426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] The token cache is being invalidate because project af02bce9aa3c48a99a340d8f94f20a1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:42.013558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:42.014497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-54c0f02c-276b-44af-a86e-c758d1cd3410 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:42.028082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 830/1655] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:41 2022] DELETE /identity/v3/projects/af02bce9aa3c48a99a340d8f94f20a1d => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:42.029184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.029654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.029730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.086816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:42.090735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:42.234125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 826/1656] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:22:42.237634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.237634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.237634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.278249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:42.279801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:42.294452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 831/1657] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:42.297967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:42.305662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:42.310135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9654a6-4a46-4011-9642-899eb36dab0b None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:42.331261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:42.332439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.332937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.333377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.334528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:42.335303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:42.337476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:42.358628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 827/1658] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:42.366415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:42.395463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a59b97-f618-437a-9009-16b528bc0b12 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:42.403876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:42.404947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.405459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.405957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.406859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:42.410281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:42.509822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 832/1659] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:42 2022] POST /identity/v3/projects => generated 363 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:42.514312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:42.541864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:42.542880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e2641a-ad67-47d7-9063-786e718062f2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:42.545627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:42.567632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:42.711982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:42.712607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.712894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.713177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.713917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:42.718312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:42.742728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf369fdc-e33c-46b2-96a6-7f81ec3f2a52 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:42.752736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:42.753644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.754196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.754764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] PATH_INFO: `/v3/projects/47a8451d52c245b0b6fe3effe4a582cf` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.756385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a8451d52c245b0b6fe3effe4a582cf)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:42.759036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:42.782981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 828/1660] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:42 2022] POST /identity/v3/users => generated 282 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:22:42.782981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:42.791145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:42.817029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10929591-b7fe-4025-afbf-7b4d0c487f87 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:42.838541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:42.854223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:42.854730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:42.855020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] PATH_INFO: `/v3/projects/423c9434d74b4c21aca451318e71c8f9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:42.859900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] RBAC: Authorizing `identity:delete_project(project_id=423c9434d74b4c21aca451318e71c8f9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:42.872597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:42.896579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a8451d52c245b0b6fe3effe4a582cf', 'request_id': 'req-02e4fc1b-9391-4a95-a472-20261e9a557b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:42.930429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a8451d52c245b0b6fe3effe4a582cf', 'request_id': 'req-02e4fc1b-9391-4a95-a472-20261e9a557b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:42.942267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a8451d52c245b0b6fe3effe4a582cf', 'request_id': 'req-02e4fc1b-9391-4a95-a472-20261e9a557b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.000933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] The token cache is being invalidate because project 47a8451d52c245b0b6fe3effe4a582cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:43.001428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.002444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.004854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 833/1661] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:42 2022] DELETE /identity/v3/projects/47a8451d52c245b0b6fe3effe4a582cf => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:43.005847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:43.007455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.065313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '423c9434d74b4c21aca451318e71c8f9', 'request_id': 'req-70981e91-33c1-4049-8b53-78a60097bf38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.088934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '423c9434d74b4c21aca451318e71c8f9', 'request_id': 'req-70981e91-33c1-4049-8b53-78a60097bf38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.145447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '423c9434d74b4c21aca451318e71c8f9', 'request_id': 'req-70981e91-33c1-4049-8b53-78a60097bf38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.158216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4fc1b-9391-4a95-a472-20261e9a557b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:43.194343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] The token cache is being invalidate because project 423c9434d74b4c21aca451318e71c8f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:43.194948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.196591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.199156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 829/1662] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:42 2022] DELETE /identity/v3/projects/423c9434d74b4c21aca451318e71c8f9 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:43.200987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:43.202869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.318272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:43.336407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70981e91-33c1-4049-8b53-78a60097bf38 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:43.357954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:43.359941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.361232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.363177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] PATH_INFO: `/v3/users/b7fa7cf30f7b4a62b90936015db276e1` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.377218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:43.377434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7fa7cf30f7b4a62b90936015db276e1)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:43.380158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.380444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.380720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.384225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:43.384818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:43.388382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:43.392161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:43.427116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 834/1663] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 421 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:43.429230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:43.435505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.473886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc027d1-6925-4af1-9705-c1449cbd6d24 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:43.500832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:43.504620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.512380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.515133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] PATH_INFO: `/v3/users/c5a5bac128e648c180843db576447937` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.517638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a5bac128e648c180843db576447937)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:43.525077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:43.529858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7fa7cf30f7b4a62b90936015db276e1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.545974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7fa7cf30f7b4a62b90936015db276e1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.546766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7fa7cf30f7b4a62b90936015db276e1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.547404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7fa7cf30f7b4a62b90936015db276e1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.558603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e5b1311-8013-4471-9e6a-4cdce0231129 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7fa7cf30f7b4a62b90936015db276e1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.601705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 830/1664] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:43 2022] DELETE /identity/v3/users/b7fa7cf30f7b4a62b90936015db276e1 => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:43.607039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebe520-7192-405e-82cd-db47404d06c3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.607341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebe520-7192-405e-82cd-db47404d06c3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.608169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebe520-7192-405e-82cd-db47404d06c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.642842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b4ebe520-7192-405e-82cd-db47404d06c3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:43.644395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b4ebe520-7192-405e-82cd-db47404d06c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.651404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a5bac128e648c180843db576447937'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.680774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a5bac128e648c180843db576447937'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.683064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a5bac128e648c180843db576447937'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.728346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a5bac128e648c180843db576447937'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.731108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a5bac128e648c180843db576447937'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:43.773544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 835/1665] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:43 2022] DELETE /identity/v3/users/c5a5bac128e648c180843db576447937 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:43.778455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:43.782236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.805412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 831/1666] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:43.807152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.807391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.808492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.847912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:43.849532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:43.980179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c932529a-c5c5-4a28-be0c-67a160cd179c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:43.994591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:43.995322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:43.995635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:43.995909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:43.997317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.003504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc69c59-d32a-4eb0-b158-43bd070fdf5b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.026311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 836/1667] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:43 2022] GET /identity/v3/roles => generated 1420 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:44.033091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.033538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.033826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.056422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 832/1668] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:44.059873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.081416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:44.082199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:44.085015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4234ce2a-60a2-42c3-8a5e-00aa0b6f989f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.100421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.101139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.101423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.101703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] PATH_INFO: `/v3/projects/d3ebac0b19024bad89291edb1db01b92/users/b622c71b8560406fbab07c32e58143bb/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.118155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 837/1669] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:44.118155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.120273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:44.137280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3ebac0b19024bad89291edb1db01b92, user_id=b622c71b8560406fbab07c32e58143bb, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.140210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.179589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:44.195478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 833/1670] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:44 2022] PUT /identity/v3/projects/d3ebac0b19024bad89291edb1db01b92/users/b622c71b8560406fbab07c32e58143bb/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:44.195478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.221433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:44.244385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfd84a6-b5f8-4347-96c5-20648d10bc1c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.313693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.314824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.315195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.315647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.329785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.331510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740a7096-9ff5-4dc7-9d93-278bfa92502a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.331713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ba0dba-07d7-4b34-bc77-84c38b9852aa admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.343286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 838/1671] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:44.344206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.344820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.345176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.345524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.346380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.347090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.347600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.348594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:44.349180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.352523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.384864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] MFA Rules not processed for user `b622c71b8560406fbab07c32e58143bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:44.389531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:44.389680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 834/1672] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:44.391263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.402616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c50042f-9458-4e65-bdef-f6122c9d8292 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.407989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.408632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.408918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.409193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.409873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.412157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.456554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 835/1673] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/projects => generated 387 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:44.458599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.562083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee3ac-aaa0-4a85-9783-f6615f41fae5 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.570962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.586279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.586766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.588739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.589651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.590897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 839/1674] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:44.592562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.597012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.622090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2e82ad-9172-470e-9c69-1be783ead442 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.645433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.646251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.646358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.646650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.647337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.649298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.656559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 836/1675] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/users => generated 294 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:44.661504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.705616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb76d6f-1986-4008-bf20-e922ed921add admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.724200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.725730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.726086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.726443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.727315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 840/1676] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/projects => generated 367 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:44.727705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.728177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.730053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.755176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 837/1677] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:22:44.761257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:44.796077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82086069-6339-4564-a00a-f86b813d163c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.921561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12921cc6-9c5d-42c3-95af-8d49f48e4183 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:44.939552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.940206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.940487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.940764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.941482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.942704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:44.943452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.969958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:44.970777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:44.981612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:44.982017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:44.982386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:44.983636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:44.985494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:44.994406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 841/1678] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:44 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:45.000444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.009817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 838/1679] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:45.013480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.025272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1322bbc-6a37-46a2-8e40-a1e9b162abdd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.033877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10fd4408-16c2-4311-803b-a316308ef3a0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.037009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.041409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.041709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.042005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.042722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.044387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.048290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.049146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.049431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.051002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] PATH_INFO: `/v3/projects/60b3308a422b4ddc8389e7fccda8e77c/users/91831f77a4374073948519a0231ea127/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.075749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 842/1680] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:44 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:45.078307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.092029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60b3308a422b4ddc8389e7fccda8e77c, user_id=91831f77a4374073948519a0231ea127, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.094741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b566875-e6c2-409f-a2e6-62d06a0864f4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.105475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f41e7a-edda-4bca-a158-ecd256ef38de admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.112524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.113313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.113603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.113889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] PATH_INFO: `/v3/projects/14ca4f3d9f4d409fa301db5953149a39/users/26b52cd802e84e5a86e2b8d36d9bb232/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.129372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 839/1681] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:45 2022] PUT /identity/v3/projects/60b3308a422b4ddc8389e7fccda8e77c/users/91831f77a4374073948519a0231ea127/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:45.142955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.142955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.142955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.204184 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ca4f3d9f4d409fa301db5953149a39, user_id=26b52cd802e84e5a86e2b8d36d9bb232, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.217149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.271856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 843/1682] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:45 2022] PUT /identity/v3/projects/14ca4f3d9f4d409fa301db5953149a39/users/26b52cd802e84e5a86e2b8d36d9bb232/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:45.272889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] MFA Rules not processed for user `91831f77a4374073948519a0231ea127`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:45.273941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:45.274874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.281370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:45.492939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 840/1683] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:45 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:45.492939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.521097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea0a7f-51de-44fb-b6fc-4ec253679c67 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.536366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.537324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.538175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.538580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] PATH_INFO: `/v3/projects/931265c8c88f415683a5689bad4e237e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.540690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=931265c8c88f415683a5689bad4e237e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.544803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.615662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f79896-b7f2-4df1-9bf6-a87f11b6583c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.623450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.624141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.624398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.624675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.627623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.629486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.661450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 841/1684] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:45 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:45.664272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.705929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d10bd16-2941-4e89-990a-e46a4b6d37f7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:45.730444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '931265c8c88f415683a5689bad4e237e', 'request_id': 'req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:45.762294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '931265c8c88f415683a5689bad4e237e', 'request_id': 'req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:45.771603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '931265c8c88f415683a5689bad4e237e', 'request_id': 'req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:45.795498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] The token cache is being invalidate because project 931265c8c88f415683a5689bad4e237e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:45.795821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:45.803930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:45.809088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 844/1685] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:45 2022] DELETE /identity/v3/projects/931265c8c88f415683a5689bad4e237e => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:45.809088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:45.809088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:45.866118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:45.944358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:45.955459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:45.955753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:45.957438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:45.958175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:45.958280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:45.960788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e7ce40-b9b3-408b-b664-de90adff1540 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:45.989362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1aa1bb-328f-4195-a0c9-adb6e446d1a4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.000723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.001000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.001269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.001556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] PATH_INFO: `/v3/projects/14ca4f3d9f4d409fa301db5953149a39/users/26b52cd802e84e5a86e2b8d36d9bb232/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.004784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14ca4f3d9f4d409fa301db5953149a39, user_id=26b52cd802e84e5a86e2b8d36d9bb232, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.015879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.020962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 842/1686] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:45 2022] POST /identity/v3/projects => generated 387 bytes in 359 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:46.023765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f148f-c305-4a31-ad91-793aa480eed9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.023996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f148f-c305-4a31-ad91-793aa480eed9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.024962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec8f148f-c305-4a31-ad91-793aa480eed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.039508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 845/1687] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:45 2022] PUT /identity/v3/projects/14ca4f3d9f4d409fa301db5953149a39/users/26b52cd802e84e5a86e2b8d36d9bb232/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:46.040523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.045254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:46.093039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ec8f148f-c305-4a31-ad91-793aa480eed9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:46.094344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ec8f148f-c305-4a31-ad91-793aa480eed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:46.251221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 843/1688] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:46.252840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.253141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.253424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.264323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e5d332-028b-4dd0-92d6-841e7d769791 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.286165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.286537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.286823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.287106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.288581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.290798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267fb28b-deaa-4bef-916d-175d99a420fc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.327700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] MFA Rules not processed for user `26b52cd802e84e5a86e2b8d36d9bb232`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:46.329572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:46.347298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 846/1689] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/users => generated 300 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:46.349269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.350069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.350579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.384624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:46.386503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:46.401154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 847/1690] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:46.402639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.421182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca694239-407a-4365-8e55-f0ce08e56a26 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.428494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.429419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.429705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.429983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.430703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.433586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.446752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 848/1691] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:46.448020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.460294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a580af14-ac89-4f6f-a19f-ede6b6a44fe6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.465439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.466592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.466592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.466784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.468036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:46.468234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.470135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.516240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 849/1692] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:46.516240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.521767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 844/1693] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:46.527027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.548714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e9f04a-5239-4d19-b68a-15430a11bdc9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.552873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f5172b-2e07-4576-8f1a-36e5a26cd025 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.559086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.560104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.560235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.560489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.561339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.563274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.572076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.574110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.578141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.578511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] PATH_INFO: `/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096/users/01585844fdff48889b331aace9e0dcc9/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.630179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 845/1694] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:46.634584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef7eec7049747a6b8c0a4bbfba06096, user_id=01585844fdff48889b331aace9e0dcc9, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.634979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.640588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.641786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:46.677136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 850/1695] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:46 2022] PUT /identity/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096/users/01585844fdff48889b331aace9e0dcc9/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:46.677136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.730158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:46.766569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a964f5e7-a22b-4b3c-9419-02ff3c369429 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.860844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951a9543-e700-44cc-9c5b-5419b5a35b10 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.873850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.874612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.874981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.875315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.878266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.882542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.907842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.908520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.908807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.909084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.922650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 851/1696] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:46 2022] POST /identity/v3/users => generated 293 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:46.924118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.938665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e0157df-a8d1-47b5-b6b9-6b05057d2a47 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:46.946477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:46.947594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:46.948033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:46.948559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:46.949383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:46.951763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:46.955390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:46.971059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 852/1697] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:46.972375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:46.989607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bde1f7-ee17-45f3-9f18-499a9e89c91a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.001069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.001909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.002196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.002482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.011249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.015460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.041773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 853/1698] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:46 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:47.042903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.056440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f60ac51-2afb-47e3-ab6a-d2270400f322 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.063575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.064493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.064921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.065660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] PATH_INFO: `/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096/users/01585844fdff48889b331aace9e0dcc9/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.068967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ef7eec7049747a6b8c0a4bbfba06096, user_id=01585844fdff48889b331aace9e0dcc9, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.073028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.109815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.111973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0349b54-26cd-45c2-940e-714ec099bd2a service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.117714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 854/1699] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:47 2022] PUT /identity/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096/users/01585844fdff48889b331aace9e0dcc9/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:47.119985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.130118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 846/1700] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:46 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:47.131947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.132379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.132796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.182567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] MFA Rules not processed for user `01585844fdff48889b331aace9e0dcc9`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:47.183840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.251618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92a56f3-b5b6-4364-82ad-33a186d04b54 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.259109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.260774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.261245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.261729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.262825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.266034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.286898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 855/1701] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:47 2022] GET /identity/v3/roles => generated 1420 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:47.291183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.322770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e17d2f-0fe7-4bac-8ca7-4a2907975c8c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.336210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.337306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.338030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.339788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] PATH_INFO: `/v3/projects/b8637b2cd2ae4a449493eac6b37e2b7f/users/e93194e173104be785c9bf628f6f402b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.347018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 847/1702] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:47.348196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.383147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8637b2cd2ae4a449493eac6b37e2b7f, user_id=e93194e173104be785c9bf628f6f402b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.395604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:47.399353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.446862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 856/1703] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:47 2022] PUT /identity/v3/projects/b8637b2cd2ae4a449493eac6b37e2b7f/users/e93194e173104be785c9bf628f6f402b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:47.446862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.447099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c130ad9e-e8f6-4a07-a870-1a301c9fc8d2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.449231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.521429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ae7450-9e67-45c5-8eb4-a3dc981003b9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:47.522515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.523299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.523706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.524073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.526792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.528072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:47.528977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.529722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.530177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.624017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:47.627026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.628484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5387f731-dc6b-4988-9c91-ebff222ac59f service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.630049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:47.631898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb37affa-0d3b-4322-8f07-bd0e92f2c926 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:47.643468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 857/1704] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:47.644829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 848/1705] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:47 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:47.645230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.646049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.646324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:47.646489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.647391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:47.647908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:47.689546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] MFA Rules not processed for user `d6e8399080a942f1b689fc8c6b42c717`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:47.693232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.712455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] MFA Rules not processed for user `e93194e173104be785c9bf628f6f402b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:47.713882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.882429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 858/1706] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:47.884148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:47.885593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:47.939195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 849/1707] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:47.944488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.007475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847f57b-3171-4470-b61b-5deff5c20b4e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.013973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.014678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.014896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.015173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.029822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.033211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.038180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:48.059356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 859/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:22:48.151098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d97bfd-9970-4687-af9c-9b74b7fe017c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.179407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.182406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.182850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.183358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.188575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.194099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.273678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 850/1709] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:47 2022] POST /identity/v3/projects => generated 385 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:22:48.278128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.299366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ee5d3cf-5d55-421b-87bc-8a6d38c15ee5 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.417009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.418367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.418661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.418942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.419749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.421974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.458362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:48.466244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.468718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:48.472903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 860/1710] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:48 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:48.473962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.499561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa83342-71df-4917-ad5e-71d149d1119d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.505702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc32224-7b3a-40f5-bddf-3e5a1142ba55 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.507193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.508416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.508991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.509501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.510669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.512824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.527911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 861/1711] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:48.532242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.545449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.546239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27f95a3-cbf3-449e-8f98-f5eeee473cb8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.548343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.548633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.548915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] PATH_INFO: `/v3/projects/eb1fb9777eab40bab0ae6d87404179aa` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.552088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.553417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.553642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.553847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] PATH_INFO: `/v3/projects/7c60330b309d4c61931f43715b73df32/users/90b204e76c8f410a970151b27bea72b7/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.558814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eb1fb9777eab40bab0ae6d87404179aa)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.567663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.578155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c60330b309d4c61931f43715b73df32, user_id=90b204e76c8f410a970151b27bea72b7, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.580907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.624923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 862/1712] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:48 2022] PUT /identity/v3/projects/7c60330b309d4c61931f43715b73df32/users/90b204e76c8f410a970151b27bea72b7/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:48.626187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.730233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aba9a3-c133-4fda-969a-63cc6d16eebc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:48.751894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eb1fb9777eab40bab0ae6d87404179aa', 'request_id': 'req-4fec7330-c936-4944-bd6d-a0805b36d890'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:48.782421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eb1fb9777eab40bab0ae6d87404179aa', 'request_id': 'req-4fec7330-c936-4944-bd6d-a0805b36d890'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:48.848427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eb1fb9777eab40bab0ae6d87404179aa', 'request_id': 'req-4fec7330-c936-4944-bd6d-a0805b36d890'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:48.872064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:48.872955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.873386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.875848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] The token cache is being invalidate because project eb1fb9777eab40bab0ae6d87404179aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:48.876176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.876662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:48.877527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4fec7330-c936-4944-bd6d-a0805b36d890 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:48.877831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:48.882244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 851/1713] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:48 2022] DELETE /identity/v3/projects/eb1fb9777eab40bab0ae6d87404179aa => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:48.890423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc7-464f-48a0-bf9f-76cc9c01b9bd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:48.890719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc7-464f-48a0-bf9f-76cc9c01b9bd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:48.894751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:48.894893 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b70bc7-464f-48a0-bf9f-76cc9c01b9bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:48.947659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 863/1714] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:48 2022] GET /identity/v3/roles => generated 1420 bytes in 321 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:48.953590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:48.956050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:48.976590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-66b70bc7-464f-48a0-bf9f-76cc9c01b9bd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:48.978242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-66b70bc7-464f-48a0-bf9f-76cc9c01b9bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:48.998587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 852/1715] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:49.001687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f36bd-af46-4e26-a01a-10148e60cb2b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.001940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.002245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.003069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.019204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.027598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.028240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.028737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] PATH_INFO: `/v3/projects/7c60330b309d4c61931f43715b73df32/users/90b204e76c8f410a970151b27bea72b7/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.033834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c60330b309d4c61931f43715b73df32, user_id=90b204e76c8f410a970151b27bea72b7, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.042305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7c1eba-fd59-45a7-8105-93dee85852c1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.057642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:49.061841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:49.083499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 864/1716] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:48 2022] PUT /identity/v3/projects/7c60330b309d4c61931f43715b73df32/users/90b204e76c8f410a970151b27bea72b7/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:49.097369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.097881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.098287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.169079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] MFA Rules not processed for user `90b204e76c8f410a970151b27bea72b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:49.170528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:49.293365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 853/1717] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 294 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:49.301772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.329128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:49.343131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6556d25-343b-4367-a34e-3f8a0268c95e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.356545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.357203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.357486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.357763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.358483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:49.358801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.365025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.411128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 865/1718] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:49 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 315 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:22:49.419005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 854/1719] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:49.426415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.427984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:49.434039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.469796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6030bb4-be6d-4ac6-8e5b-8ca83a7db9c8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.484932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.490200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.490916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.491386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.496090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.498896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.526467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:49.549318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184b85d5-df84-4217-98fc-612d1f1843ca None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.554833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.555758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.556229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.556673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.567194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 855/1720] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:49 2022] POST /identity/v3/projects => generated 367 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:49.569593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.572981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.577781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.591131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 866/1721] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:49 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:49.628967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.632564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:49.689425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06b3a51-c9f4-4358-8b0b-471fd360c4d4 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.705523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.706200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.706482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.712294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] PATH_INFO: `/v3/users/91831f77a4374073948519a0231ea127` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.716714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91831f77a4374073948519a0231ea127)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.719713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.723203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26154bb-5b1c-4acd-9b62-732fb68dbad0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.735554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.738534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.739004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.740228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.741392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.746986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.811728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 856/1722] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:49 2022] POST /identity/v3/users => generated 284 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:49.811728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.847343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91831f77a4374073948519a0231ea127'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:49.853891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b697f77-2b15-4799-95c3-0c432119fcf8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:49.873836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:49.874861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:49.875165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:49.876307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] PATH_INFO: `/v3/users/e93194e173104be785c9bf628f6f402b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:49.879302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e93194e173104be785c9bf628f6f402b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:49.885427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:49.887451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91831f77a4374073948519a0231ea127'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:49.888476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91831f77a4374073948519a0231ea127'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:49.913832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91831f77a4374073948519a0231ea127'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:49.915068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91831f77a4374073948519a0231ea127'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:49.959439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 867/1723] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:49 2022] DELETE /identity/v3/users/91831f77a4374073948519a0231ea127 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:49.960479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:49.962421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.007970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e93194e173104be785c9bf628f6f402b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:50.030756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e93194e173104be785c9bf628f6f402b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:50.031881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e93194e173104be785c9bf628f6f402b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:50.032198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e93194e173104be785c9bf628f6f402b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:50.051367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5eb9a030-857b-4939-9145-8b0b7aaeea7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e93194e173104be785c9bf628f6f402b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:50.078775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 857/1724] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:49 2022] DELETE /identity/v3/users/e93194e173104be785c9bf628f6f402b => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:50.080478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.080770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.081069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.093226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:50.106625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eea860f-7720-43b1-87b3-0a67621a8372 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:50.118827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:50.121320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.196508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:50.198626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.199095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.200421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.201727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:50.203706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bdd83a-f42b-45bc-930b-2805b1b0501e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:50.210048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:50.235892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 868/1725] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:49 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:50.236560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 858/1726] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:50.237892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:50.241546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.241843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.242127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.243455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.264295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:50.266930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.273041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2e711c-1e38-44ae-988d-f122d3c28016 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:50.275185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 869/1727] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:50.292640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:50.293296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.293581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.294115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.294560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:50.296407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:50.302272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:50.307296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.314276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 859/1728] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:50 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:50.315237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:50.330054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c1b7c-2e29-4462-9b15-3a9db4ba2a66 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:50.336316 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:50.337134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.337488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.337838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] PATH_INFO: `/v3/projects/75a4e048fae34fe898c5fea4f7e683cd/users/3b548e89ee7946e58199b0b7b5fb14c5/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.370968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75a4e048fae34fe898c5fea4f7e683cd, user_id=3b548e89ee7946e58199b0b7b5fb14c5, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:50.373769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:50.402984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 860/1729] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:50 2022] PUT /identity/v3/projects/75a4e048fae34fe898c5fea4f7e683cd/users/3b548e89ee7946e58199b0b7b5fb14c5/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:50.405634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:50.407554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.481560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccc147e8-cf78-4272-b9c1-4136878b789c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:50.486507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:50.487254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.487549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.487832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.499192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c2b036-66d0-4ada-bbb0-3a3c5c42c8f8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:50.504738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:50.505371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.505657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.506466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.637950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:50.640520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bab0262-1027-4bdf-96d7-d75189054071 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:50.646800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:50.648818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:50.668802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 870/1730] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:50.670448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:50.670904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:50.671459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:50.674109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 861/1731] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:50.709552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] MFA Rules not processed for user `3b548e89ee7946e58199b0b7b5fb14c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:50.711269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:50.766456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:50.849503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 871/1732] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:50 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:50.865558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:50.867672 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:51.001182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0790728-a71a-479d-ba16-3a2f13964302 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:51.006666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:51.007651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:51.008148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:51.008633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:51.029850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:51.031740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:51.059994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 862/1733] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:50 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:52.242866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:52.244460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:52.269074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe445d54-a966-4a56-be66-8a97f29808db None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:52.279634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:52.281201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:52.281495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:52.283052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] PATH_INFO: `/v3/projects/b8637b2cd2ae4a449493eac6b37e2b7f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:52.285759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8637b2cd2ae4a449493eac6b37e2b7f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:52.292535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:52.380018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8637b2cd2ae4a449493eac6b37e2b7f', 'request_id': 'req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.409953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8637b2cd2ae4a449493eac6b37e2b7f', 'request_id': 'req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.423339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8637b2cd2ae4a449493eac6b37e2b7f', 'request_id': 'req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.441823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] The token cache is being invalidate because project b8637b2cd2ae4a449493eac6b37e2b7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:52.442356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.443186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3adaeaba-774c-4b4d-af8b-f913cc8d0a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.448367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 872/1734] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:52 2022] DELETE /identity/v3/projects/b8637b2cd2ae4a449493eac6b37e2b7f => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:52.450296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:52.464445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a383c7fb-ee6c-4f00-82f6-9560fd9f3a48 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:52.480564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:52.622591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:52.630208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:52.630530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:52.634457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] PATH_INFO: `/v3/users/90b204e76c8f410a970151b27bea72b7` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:52.644776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] RBAC: Authorizing `identity:delete_user(user_id=90b204e76c8f410a970151b27bea72b7)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:52.665178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:52.900849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '90b204e76c8f410a970151b27bea72b7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.953699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '90b204e76c8f410a970151b27bea72b7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.955145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '90b204e76c8f410a970151b27bea72b7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.957488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '90b204e76c8f410a970151b27bea72b7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:52.967268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:52.967594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:52.967919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:53.009731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '90b204e76c8f410a970151b27bea72b7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.046233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] MFA Rules not processed for user `d6e8399080a942f1b689fc8c6b42c717`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:53.064139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:53.074379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 863/1735] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:52 2022] DELETE /identity/v3/users/90b204e76c8f410a970151b27bea72b7 => generated 0 bytes in 625 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:53.270371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:53.272545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 873/1736] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:53.274370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:53.458038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d95722d-b090-45b7-a219-a5714a8c82dd admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:53.472161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:53.472960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:53.473372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:53.473780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] PATH_INFO: `/v3/projects/60b3308a422b4ddc8389e7fccda8e77c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:53.478787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] RBAC: Authorizing `identity:delete_project(project_id=60b3308a422b4ddc8389e7fccda8e77c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:53.481130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:53.581861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:53.589595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60b3308a422b4ddc8389e7fccda8e77c', 'request_id': 'req-9dc78aa7-daf5-4361-a381-49a22417ccdc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.608479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60b3308a422b4ddc8389e7fccda8e77c', 'request_id': 'req-9dc78aa7-daf5-4361-a381-49a22417ccdc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.625526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60b3308a422b4ddc8389e7fccda8e77c', 'request_id': 'req-9dc78aa7-daf5-4361-a381-49a22417ccdc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.644789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] The token cache is being invalidate because project 60b3308a422b4ddc8389e7fccda8e77c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:53.645218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.647823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9dc78aa7-daf5-4361-a381-49a22417ccdc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.653842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:53.654471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 864/1737] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:53 2022] DELETE /identity/v3/projects/60b3308a422b4ddc8389e7fccda8e77c => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:53.669903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21166b0-b580-45b7-8bcf-92faea56f1d2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:53.801654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:53.802452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:53.803454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:53.803862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] PATH_INFO: `/v3/users/01585844fdff48889b331aace9e0dcc9` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:53.805654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01585844fdff48889b331aace9e0dcc9)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:53.815682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:53.818944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:53.923276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01585844fdff48889b331aace9e0dcc9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.953081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01585844fdff48889b331aace9e0dcc9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.953645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01585844fdff48889b331aace9e0dcc9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.977294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01585844fdff48889b331aace9e0dcc9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:53.978138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-554e4954-3b5d-49f3-8fbb-a4c17f02eb71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01585844fdff48889b331aace9e0dcc9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:54.001458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 874/1738] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:53 2022] DELETE /identity/v3/users/01585844fdff48889b331aace9e0dcc9 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:55.408148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:55.408677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:55.408677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:55.410343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 865/1739] 10.222.0.40 () {54 vars in 912 bytes} [Sun Jul 31 14:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:55.426011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:55.426887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:55.427327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:55.548406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:55.550761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:55.801006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 875/1740] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:22:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 376 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:22:55.892684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:55.894466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:55.930899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d6d45-59bc-4226-b047-9c09b1837633 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:55.953608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:55.955263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:55.957922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:55.958379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:55.978316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:55.979813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:56.002340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:56.004089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f553fe7-598f-4b09-8850-487d303cb74a service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:56.060943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 866/1741] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:22:56.216223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ade88b3-5b86-410f-8669-2b43c95f8502 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:56.235518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:56.236492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:56.236975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:56.237593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] PATH_INFO: `/v3/users/b622c71b8560406fbab07c32e58143bb` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:56.239299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b622c71b8560406fbab07c32e58143bb)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:56.242682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:56.289805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:56.360939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b622c71b8560406fbab07c32e58143bb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.399613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b622c71b8560406fbab07c32e58143bb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.400579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b622c71b8560406fbab07c32e58143bb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.417983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b622c71b8560406fbab07c32e58143bb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.418631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b622c71b8560406fbab07c32e58143bb'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.443027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 876/1742] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:55 2022] DELETE /identity/v3/users/b622c71b8560406fbab07c32e58143bb => generated 0 bytes in 465 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:22:56.447574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:56.448571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:56.448995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:56.471093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:56.472979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:56.504749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:56.506055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:56.660285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 867/1743] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:56.713989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b2ee95-9bdc-431b-92a1-1f0a70cd7a5f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:56.723006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:56.726057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:56.736753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:56.737329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:56.737619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:56.737892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] PATH_INFO: `/v3/projects/7c60330b309d4c61931f43715b73df32` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:56.741976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c60330b309d4c61931f43715b73df32)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:56.744236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:56.926276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2d6799-d1f2-4961-b691-7b5ae1895610 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:56.934517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c60330b309d4c61931f43715b73df32', 'request_id': 'req-655b0049-da24-47dc-9481-fd128e775af2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.967101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c60330b309d4c61931f43715b73df32', 'request_id': 'req-655b0049-da24-47dc-9481-fd128e775af2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:56.980383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c60330b309d4c61931f43715b73df32', 'request_id': 'req-655b0049-da24-47dc-9481-fd128e775af2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.007194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] The token cache is being invalidate because project 7c60330b309d4c61931f43715b73df32 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:57.007629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.008317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.010723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 877/1744] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:56 2022] DELETE /identity/v3/projects/7c60330b309d4c61931f43715b73df32 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:57.011652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:57.013017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.065280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:57.065866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.066340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.066450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.072277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.081997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:57.121896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:57.184627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b0049-da24-47dc-9481-fd128e775af2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:57.190803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:57.191613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.192013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.194106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] PATH_INFO: `/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.195954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ef7eec7049747a6b8c0a4bbfba06096)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:57.205285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:57.266974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:57.268900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc7e566-99fc-454c-b1f0-dc9043e14fb5 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:57.296269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 868/1745] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 573 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:57.299164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264bf6-e830-4154-a8ae-c0f1994429ec None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.300761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264bf6-e830-4154-a8ae-c0f1994429ec None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.301843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5264bf6-e830-4154-a8ae-c0f1994429ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.308653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ef7eec7049747a6b8c0a4bbfba06096', 'request_id': 'req-5c477458-ad8d-4e15-ab9f-0793790b4644'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.325450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ef7eec7049747a6b8c0a4bbfba06096', 'request_id': 'req-5c477458-ad8d-4e15-ab9f-0793790b4644'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.335454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ef7eec7049747a6b8c0a4bbfba06096', 'request_id': 'req-5c477458-ad8d-4e15-ab9f-0793790b4644'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.344510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c5264bf6-e830-4154-a8ae-c0f1994429ec None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:57.346129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c5264bf6-e830-4154-a8ae-c0f1994429ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.354403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] The token cache is being invalidate because project 0ef7eec7049747a6b8c0a4bbfba06096 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:57.355709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.359443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5c477458-ad8d-4e15-ab9f-0793790b4644 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:57.383027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 878/1746] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:57 2022] DELETE /identity/v3/projects/0ef7eec7049747a6b8c0a4bbfba06096 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:57.389105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ded9-e4be-400d-9c14-748926e14746 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.389504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ded9-e4be-400d-9c14-748926e14746 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.390171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e03ded9-e4be-400d-9c14-748926e14746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.477168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0e03ded9-e4be-400d-9c14-748926e14746 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:57.483366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0e03ded9-e4be-400d-9c14-748926e14746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.556858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 869/1747] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:57.559312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.559629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.559960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.633197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] MFA Rules not processed for user `53dab2ec3f2e4973893fb838c3d1d545`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:57.634495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.674862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 879/1748] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:22:57.674862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437cba9-3ffc-4661-99c6-3ec91bb57911 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.674862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437cba9-3ffc-4661-99c6-3ec91bb57911 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.674862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437cba9-3ffc-4661-99c6-3ec91bb57911 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.721647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4437cba9-3ffc-4661-99c6-3ec91bb57911 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:57.727129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4437cba9-3ffc-4661-99c6-3ec91bb57911 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.752427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 880/1749] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:57.757605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.758605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.758897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.762567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 870/1750] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:57.764852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:57.820805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4203afe-867e-4be2-b978-8b91594cd768 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:57.830940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:57.832899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.834434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:57.835073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.835555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.835835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.836561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:57.837364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:57.839669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:57.861454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 881/1751] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:57.862102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:57.866078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:57.886354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 871/1752] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:57.886354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:57.900707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19dd8cf2-699b-44c7-af2f-3b4e1552f449 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:57.926492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:57.927157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:57.927513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:57.928292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:57.930125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:22:57.930343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:57.932277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:57.964933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 872/1753] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:22:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:22:57.964933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:57.981386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb95f7-27c1-44b1-9227-68c92ffd3155 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:57.999969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.000198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:58.001137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.001671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.002148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.003044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.007234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4e2233-4647-49cf-b8ad-7cce0c4e4599 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.066137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2231f182-2361-4067-89f8-0a104cfc0daf None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.068249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:58.093336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 873/1754] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:57 2022] POST /identity/v3/projects => generated 379 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:58.093336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b66f-6fc6-438d-863d-26be916443b9 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.093336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b66f-6fc6-438d-863d-26be916443b9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.093336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a24b66f-6fc6-438d-863d-26be916443b9 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.106729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 874/1755] 10.222.0.40 () {54 vars in 961 bytes} [Sun Jul 31 14:22:58 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:58.108461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b66f-6fc6-438d-863d-26be916443b9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.117614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.119738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.120143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.120500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.134713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.138639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec0e527d-63f5-4fcd-9c66-7f3ebd6206ab service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.160249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 882/1756] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:58.162740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24088-458a-437f-9f41-768cab69757e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.163233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24088-458a-437f-9f41-768cab69757e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.163685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf24088-458a-437f-9f41-768cab69757e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.225480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1bf24088-458a-437f-9f41-768cab69757e None None] MFA Rules not processed for user `db0c001088aa4c86b4562c369deb6c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:58.226770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1bf24088-458a-437f-9f41-768cab69757e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:58.306328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a24b66f-6fc6-438d-863d-26be916443b9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.311802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 883/1757] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:58.312698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf24088-458a-437f-9f41-768cab69757e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.325554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.325999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf24088-458a-437f-9f41-768cab69757e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.326792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.328289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.330815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.331679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.332791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.334825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.335565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.335974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.336996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.339190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32a75fe-3408-49a5-bc5c-0eb7d560beb7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.343446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760eb733-1296-4770-a0c6-e590a3f40eaf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.376237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 884/1758] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/users => generated 290 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:58.377866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.378342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.378819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.407379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:58.410002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:58.428124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 875/1759] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/projects => generated 383 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:58.428124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.428124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.428124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.497494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] MFA Rules not processed for user `db0c001088aa4c86b4562c369deb6c46`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:58.498869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:58.591888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 885/1760] 10.222.0.40 () {58 vars in 1058 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:58.592692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.613622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d39382-fcab-43ca-bac5-82b717a43136 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.642166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 876/1761] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:58.643969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.710482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:58.764814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.765648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.768835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.768835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.768835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.770054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.795601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca6555d-81b0-4271-85df-24e4ad54c200 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.795863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 886/1762] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:58 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:58.796736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.805959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.806878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.807337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.807851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.808710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:58.810827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:58.875099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 877/1763] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:22:58 2022] POST /identity/v3/users => generated 292 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:22:58.875099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:58.919952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:58.947222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e8c344-ab18-4f6f-bc74-0582b9f3ae0d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:58.957794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:58.959237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:58.959738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:58.960019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:58.962322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:58.998936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.001047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.017455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 887/1764] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:59.018385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.019987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.023191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef58daa-5b48-4e8e-880d-4879f6d69fc1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.029349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.030470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.030470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.030663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.040304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.041570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.063072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 878/1765] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:22:59.064106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.067573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bbc70d-f73e-4a20-841a-bcbda369c598 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.077576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.078276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.078560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.078852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] PATH_INFO: `/v3/projects/d3ebac0b19024bad89291edb1db01b92` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.080191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3ebac0b19024bad89291edb1db01b92)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.091263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.093646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c90193b-1218-4c94-ba66-13398379808f service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.102473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.103724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.104750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.105278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.106237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.110039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02014aa4-810c-4b5c-860a-fefa49b89eed admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.133394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 879/1766] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:59.135844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.136785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.137352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.198083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] MFA Rules not processed for user `10a4469c46f64feb819f3b19a0f4e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:59.201462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.220641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3ebac0b19024bad89291edb1db01b92', 'request_id': 'req-07475643-5bea-4116-a0d2-3dd4e93a7241'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.242083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3ebac0b19024bad89291edb1db01b92', 'request_id': 'req-07475643-5bea-4116-a0d2-3dd4e93a7241'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.263693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3ebac0b19024bad89291edb1db01b92', 'request_id': 'req-07475643-5bea-4116-a0d2-3dd4e93a7241'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.280588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] The token cache is being invalidate because project d3ebac0b19024bad89291edb1db01b92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:22:59.280887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.281576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.286079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 888/1767] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:22:59 2022] DELETE /identity/v3/projects/d3ebac0b19024bad89291edb1db01b92 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:59.291044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.293750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.361823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 880/1768] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:59.362857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.365684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.408724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:59.448817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07475643-5bea-4116-a0d2-3dd4e93a7241 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.468584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.469242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.469526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.469803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.471123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.471591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2175c4-0286-4b92-8232-221c0d7e2bab None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.478282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.478451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.479040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.479324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.479687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] PATH_INFO: `/v3/projects/d739e0e346a1442f80113f14e1b4a603/users/a5ac1d9a881747de93aa411524e1015b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.501817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d739e0e346a1442f80113f14e1b4a603, user_id=a5ac1d9a881747de93aa411524e1015b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.504182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 889/1769] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:59.505836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9978e3-caf6-4ff4-ba27-f5e921aa2ee0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.508421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.510717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.529612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 881/1770] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:22:59 2022] PUT /identity/v3/projects/d739e0e346a1442f80113f14e1b4a603/users/a5ac1d9a881747de93aa411524e1015b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:59.531100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.531592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.532803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.534611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af472c3-e570-4c1c-bed1-24537c74b1b3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.567252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:22:59.585274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] MFA Rules not processed for user `10a4469c46f64feb819f3b19a0f4e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:59.587278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.631315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.632021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.632494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.632597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] PATH_INFO: `/v3/users/26b52cd802e84e5a86e2b8d36d9bb232` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.634201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] RBAC: Authorizing `identity:delete_user(user_id=26b52cd802e84e5a86e2b8d36d9bb232)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.640743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.702959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 882/1771] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:22:59.704652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.734540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26b52cd802e84e5a86e2b8d36d9bb232'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.737572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73047efe-5095-481a-a5ff-5f11a7cc9217 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:22:59.746351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:22:59.747003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.747282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.747684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.748521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:22:59.750695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7ca488-21fb-4a39-8a22-deaca91324fe admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:22:59.762533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26b52cd802e84e5a86e2b8d36d9bb232'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.763874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26b52cd802e84e5a86e2b8d36d9bb232'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.773375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 883/1772] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:22:59 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:22:59.776048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.776433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.776768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.781237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26b52cd802e84e5a86e2b8d36d9bb232'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.782836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-58fffdc9-4ac6-450e-8f9c-7e21c9c68230 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26b52cd802e84e5a86e2b8d36d9bb232'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:22:59.803298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 890/1773] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:22:59 2022] DELETE /identity/v3/users/26b52cd802e84e5a86e2b8d36d9bb232 => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:22:59.803691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:22:59.805375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:22:59.805375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:22:59.811910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 891/1774] 10.222.0.40 () {54 vars in 912 bytes} [Sun Jul 31 14:22:59 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:22:59.812924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:22:59.815689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.843064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] MFA Rules not processed for user `a5ac1d9a881747de93aa411524e1015b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:22:59.846751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:22:59.976794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:00.006456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 884/1775] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:22:59 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:00.006456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.009529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.012262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c3f777-9034-4200-a57e-7e5b68ba9329 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.026448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.027296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.027708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.028547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.030718 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.146052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.149269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e43cf1-f3e4-4586-89e5-180134e8acb1 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.165769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 892/1776] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:22:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:00.168326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219051-8b80-44dc-9ce3-9e9bf29167b6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.168624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219051-8b80-44dc-9ce3-9e9bf29167b6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.168900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219051-8b80-44dc-9ce3-9e9bf29167b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.202983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d0219051-8b80-44dc-9ce3-9e9bf29167b6 None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:00.204507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d0219051-8b80-44dc-9ce3-9e9bf29167b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.215368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c375c23-5cd7-4a2b-ad89-c40c53aa4613 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.222194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.223840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.224490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.225076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] PATH_INFO: `/v3/projects/49112a7a3839497aa5a24e26f6410387/users/fa6c836585314c77a80459102cc1c84d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.276317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] RBAC: Authorizing `identity:create_grant(project_id=49112a7a3839497aa5a24e26f6410387, user_id=fa6c836585314c77a80459102cc1c84d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.283930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.317332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 885/1777] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:00 2022] PUT /identity/v3/projects/49112a7a3839497aa5a24e26f6410387/users/fa6c836585314c77a80459102cc1c84d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:00.320054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.322211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.336766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 893/1778] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:00.339258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.339680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.340689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.341435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:00.404946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] MFA Rules not processed for user `fa6c836585314c77a80459102cc1c84d`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:00.406201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.469605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed7fddc-0d39-48a3-85c6-eadf4cfd8fce admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.477907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.478733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.479091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.479532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.480570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.484087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.544845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 886/1779] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/projects => generated 381 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:00.544845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.548570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:00.639723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 894/1780] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 302 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:00.643172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.658846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b207c713-1876-4d10-9196-9d6015ea3177 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.718235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3ecbcf-162e-4e46-a974-c95cdf07b345 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.721238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:00.729488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.732618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.733865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.734292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.757002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.759155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.765017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.765810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.766146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.766415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.767281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.769757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.778581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 887/1781] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:00.779593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.821768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 895/1782] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/users => generated 297 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:00.826469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.827840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d28bb6-303e-4868-bc7e-1ebce1e84f34 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.839641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.840750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.841024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.841302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.844087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.846098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.853160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd9a03d-10f1-4180-8a0f-c55dbe7cb480 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.868833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.872153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.874198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.874805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:00.877438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:00.883547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:00.907799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 896/1783] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:00 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:00.912494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.941142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1fb4d-63a9-4619-aec2-f46716b3cdcf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:00.962677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 888/1784] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:00.963701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:00.964064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:00.964905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:00.965935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:00.966207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] PATH_INFO: `/v3/projects/bd52358d165a4431aff712c4ee62317a/users/426e7bf4d5014becb8e044d3a7e158c5/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.014824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd52358d165a4431aff712c4ee62317a, user_id=426e7bf4d5014becb8e044d3a7e158c5, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.018626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.059932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 897/1785] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:00 2022] PUT /identity/v3/projects/bd52358d165a4431aff712c4ee62317a/users/426e7bf4d5014becb8e044d3a7e158c5/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:01.061958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:01.066682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:01.166305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4420e8ba-e056-4593-a329-3b7c5ea66b2b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.206247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:01.261273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac1d78-75c1-4d68-9e71-c25ee8fa6fd0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.273044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:01.273756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.274041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.274322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] PATH_INFO: `/v3/projects/14ca4f3d9f4d409fa301db5953149a39` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.276442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=14ca4f3d9f4d409fa301db5953149a39)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.278694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.345708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:01.346879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.350464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.350833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.353327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.375573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.421820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14ca4f3d9f4d409fa301db5953149a39', 'request_id': 'req-18c49180-ec17-400b-bc3e-c99a6df56ca9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:01.442540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14ca4f3d9f4d409fa301db5953149a39', 'request_id': 'req-18c49180-ec17-400b-bc3e-c99a6df56ca9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:01.457845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14ca4f3d9f4d409fa301db5953149a39', 'request_id': 'req-18c49180-ec17-400b-bc3e-c99a6df56ca9'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:01.475777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 889/1786] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:00 2022] POST /identity/v3/users => generated 297 bytes in 513 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:01.480553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:01.507900 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] The token cache is being invalidate because project 14ca4f3d9f4d409fa301db5953149a39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:01.508217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:01.508942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:01.514117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 898/1787] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:01 2022] DELETE /identity/v3/projects/14ca4f3d9f4d409fa301db5953149a39 => generated 0 bytes in 450 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:01.515004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:01.516603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aeba26-c851-42e8-9015-048b7b3086c2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.517129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:01.560741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:01.635047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c49180-ec17-400b-bc3e-c99a6df56ca9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.641251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:01.642032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.642392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.642775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.643968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.648516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78d77b7-8c4b-40a7-a1b2-eaf5c6382f7b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.669072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 899/1788] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:01.669072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.669072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.669072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.729250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:01.731217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:01.786415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:01.787774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.788056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.788490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.789433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.792291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a38bd-657b-4b3f-92e8-cf9ff29c24b5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.792882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 900/1789] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:01.799522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:01.815399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 890/1790] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:01 2022] GET /identity/v3/roles => generated 1420 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:01.816810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.817155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.817440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.830832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822a13f1-f819-4bab-924c-48cb449f7378 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.841521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:01.842535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:01.843062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:01.843551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] PATH_INFO: `/v3/projects/92d8851648184bca808adc9d8b3dfcc8/users/7399f5d53c4c4c40baa14e919d5accd0/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:01.844155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:01.846143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:01.861223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 891/1791] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:01.863374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:01.866032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:01.876843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d8851648184bca808adc9d8b3dfcc8, user_id=7399f5d53c4c4c40baa14e919d5accd0, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:01.881742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:01.915099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 901/1792] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:01 2022] PUT /identity/v3/projects/92d8851648184bca808adc9d8b3dfcc8/users/7399f5d53c4c4c40baa14e919d5accd0/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:01.919437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee67abcf-5238-4cb6-8380-5b6544c47886 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:01.919679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:02.080735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:02.081729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.082116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.082498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] PATH_INFO: `/v3/projects/bd52358d165a4431aff712c4ee62317a/users/426e7bf4d5014becb8e044d3a7e158c5/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.087223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd52358d165a4431aff712c4ee62317a, user_id=426e7bf4d5014becb8e044d3a7e158c5, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:02.092095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:02.117779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 892/1793] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:01 2022] PUT /identity/v3/projects/bd52358d165a4431aff712c4ee62317a/users/426e7bf4d5014becb8e044d3a7e158c5/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:02.120188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:02.191624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ce6833-7161-4437-a540-a186db111d01 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:02.245294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:02.297503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fc430f-878c-4286-ba70-b036fe21c9e6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:02.312172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:02.313427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.313821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.316105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.317216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:02.321177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0870412b-4f06-4c6d-a4d6-e202b3c03f05 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:02.348617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:02.349324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.349660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.349942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.350658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:02.350967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:02.352607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:02.394131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 893/1794] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:02 2022] GET /identity/v3/roles => generated 1420 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:02.403498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.405552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.408035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.415348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 902/1795] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 499 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:02.415839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:02.462024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0e8aed-04ae-4872-bb61-f3948a6fb525 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:02.475247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:02.476034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.476420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.476712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] PATH_INFO: `/v3/projects/92d8851648184bca808adc9d8b3dfcc8/users/7399f5d53c4c4c40baa14e919d5accd0/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.483604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=92d8851648184bca808adc9d8b3dfcc8, user_id=7399f5d53c4c4c40baa14e919d5accd0, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:02.486446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:02.519780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 903/1796] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:02 2022] PUT /identity/v3/projects/92d8851648184bca808adc9d8b3dfcc8/users/7399f5d53c4c4c40baa14e919d5accd0/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:02.521183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:02.554448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] MFA Rules not processed for user `426e7bf4d5014becb8e044d3a7e158c5`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:02.555722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:02.599347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:02.777819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c05481-c5e3-4490-965d-61ead60cc0cb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:02.788705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:02.789447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.789810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.790159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.790921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:02.792988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e873e-367b-4bcc-85d7-96ac1fc9cb2a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:02.854424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 904/1797] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:02 2022] POST /identity/v3/projects => generated 371 bytes in 334 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:02.855939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4fdd9-37ca-4319-82ae-38a1d8dba2bd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:02.856235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4fdd9-37ca-4319-82ae-38a1d8dba2bd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:02.858667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f4fdd9-37ca-4319-82ae-38a1d8dba2bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:02.871551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 894/1798] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:02.879979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:02.885954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:02.925736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-79f4fdd9-37ca-4319-82ae-38a1d8dba2bd None None] MFA Rules not processed for user `7399f5d53c4c4c40baa14e919d5accd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:02.929178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-79f4fdd9-37ca-4319-82ae-38a1d8dba2bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:02.941957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce2b434a-5245-4708-ac6e-60691e929fd7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:03.101968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 905/1799] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:02 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:03.104740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4450-40f2-4889-97d6-72a93ecafbf3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.105041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4450-40f2-4889-97d6-72a93ecafbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.105546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004a4450-40f2-4889-97d6-72a93ecafbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.116437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:03.118782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.119183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.119971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] PATH_INFO: `/v3/users/3b548e89ee7946e58199b0b7b5fb14c5` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.123505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b548e89ee7946e58199b0b7b5fb14c5)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:03.129114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:03.153477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-004a4450-40f2-4889-97d6-72a93ecafbf3 None None] MFA Rules not processed for user `10a4469c46f64feb819f3b19a0f4e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:03.154813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-004a4450-40f2-4889-97d6-72a93ecafbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.269810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b548e89ee7946e58199b0b7b5fb14c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:03.297682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:03.309674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b548e89ee7946e58199b0b7b5fb14c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:03.311461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 906/1800] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:03.312763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.313071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.313460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.315728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b548e89ee7946e58199b0b7b5fb14c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:03.316476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b548e89ee7946e58199b0b7b5fb14c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:03.332849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b548e89ee7946e58199b0b7b5fb14c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:03.359357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 895/1801] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:02 2022] DELETE /identity/v3/users/3b548e89ee7946e58199b0b7b5fb14c5 => generated 0 bytes in 487 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:03.363042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:03.364504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.439104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:03.462196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:03.465998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.590392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48099490-f40c-4fd4-bb99-f6e4fca819cc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:03.611480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:03.612473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.616529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.617308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.620274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:03.626389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:03.678199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 896/1802] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:03 2022] POST /identity/v3/users => generated 286 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:03.679141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:03.681999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.707912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 907/1803] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:23:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:03.710072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:03.714007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.825315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49ce21a-c821-4730-a2ee-c59efae2cec4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:03.831272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:03.832897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.833420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.835813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.838985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:03.905433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f9a1fe8-c5c4-4555-93e4-282a3a1ad36a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:03.919227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:03.920605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:03.920936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:03.921211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:03.929648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.000817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.006566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c983787c-4c6c-4c62-bc52-f7bc78694e3f service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.032415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 897/1804] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:04.032958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.033830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.034118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.034394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.035415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adec45ff-9983-4f48-83a7-504589986ae7 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.069083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 908/1805] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:03 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:04.070640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33de8-d093-4c25-b788-137daba690da None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.071293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33de8-d093-4c25-b788-137daba690da None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.072311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa33de8-d093-4c25-b788-137daba690da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.075999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] MFA Rules not processed for user `53dab2ec3f2e4973893fb838c3d1d545`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:04.077430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.104813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=2,keystone:SELECT=94,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:04.128472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0fa33de8-d093-4c25-b788-137daba690da None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:04.129729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0fa33de8-d093-4c25-b788-137daba690da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.154383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 909/1806] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:04.160366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa33de8-d093-4c25-b788-137daba690da None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.183618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa33de8-d093-4c25-b788-137daba690da None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:04.200301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:04.201042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.201400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.201752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.204599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.206376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.218169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 898/1807] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:04.237094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.243885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.244715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 910/1808] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:04.245884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.251939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.409495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:04.475737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e6e506-158a-4a76-8e75-875593c9b72a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:04.484168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:04.485854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.486506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.488617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.532854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.539776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.586555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 911/1809] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:04.589021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.590861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5357f450-bc0d-4573-9225-7fe7e61e5c6c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:04.614690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:04.616739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.617293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.617894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b666663-5f36-4370-b4a6-b714c8d0df12 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:04.618402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.631720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.634030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.634260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:04.635172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.635536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.636002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.638026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.641007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d954e8e5-305f-4fec-b36e-3dc59b335082 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.649739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 899/1810] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:23:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:23:04.652851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.664484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5af5df-bbde-45b6-b1cc-9d2a39514e1a service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:04.668946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:04.669802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.670703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.670703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.672445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.733393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 912/1811] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:04 2022] GET /identity/v3/roles => generated 1420 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:04.743288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:04.745078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:04.745385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:04.863925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:04.873930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:04.917371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 913/1812] 10.222.0.17 () {58 vars in 1015 bytes} [Sun Jul 31 14:23:04 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:04.918882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:04.937721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:04.944375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:04.973736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 900/1813] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:04 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:05.004224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dfe8f7-5018-4d75-8a62-26d7edbba988 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.033218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.033743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.034241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.034441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.034816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] PATH_INFO: `/v3/projects/479c4bda3bdc41dcb617b778c52deb20/users/95e33684ef7f4712a9dc517b28d29071/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.057888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4d012b-4464-4247-8ab3-d5b3ead692fb service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.071720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426e7bf4d5014becb8e044d3a7e158c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd52358d165a4431aff712c4ee62317a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.072378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.072974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.073269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.075608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:05.075879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.087680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.119249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=479c4bda3bdc41dcb617b778c52deb20, user_id=95e33684ef7f4712a9dc517b28d29071, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.126270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.163144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 901/1814] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:05.164233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.168397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:05.184995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 914/1815] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:04 2022] PUT /identity/v3/projects/479c4bda3bdc41dcb617b778c52deb20/users/95e33684ef7f4712a9dc517b28d29071/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:05.186086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.221762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc3d905e-27f3-4f79-b634-ae7bd523c939 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.276217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:05.384282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.385796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.386185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.387719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.397496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f08847-dfe3-4852-84a2-069c1f976bdd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.406035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.408858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12426f-c0db-43ec-97cf-a23274652707 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.414623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426e7bf4d5014becb8e044d3a7e158c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd52358d165a4431aff712c4ee62317a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.415494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.415884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.416260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.418304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.421187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.433597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 902/1816] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:05 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:05.435233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.436630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.437518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.490552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:05.492816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 915/1817] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:05 2022] POST /identity/v3/projects => generated 354 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:05.493793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:05.496600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.651247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 903/1818] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:05.656050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0cc50a-e623-4b92-b517-9e8392d635ad tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.667383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426e7bf4d5014becb8e044d3a7e158c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd52358d165a4431aff712c4ee62317a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.667651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.667863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.668122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.668858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:05.669162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.671027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.684844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 916/1819] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:05.690065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.775560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:05.800448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae6a865-f976-43d6-92ea-a5a97eeb11f6 tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.833397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '426e7bf4d5014becb8e044d3a7e158c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'bd52358d165a4431aff712c4ee62317a', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.839584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.839971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.841763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] PATH_INFO: `/v3/projects/518bcc4264ff4400a2a71e21d813a91c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:05.848031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorizing `identity:delete_project(project_id=518bcc4264ff4400a2a71e21d813a91c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:05.865833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:05.933848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:05.968544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a58f0b-0eb2-46aa-93c7-0085943cec1d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:05.982702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:05.989930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:05.990535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:05.991147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:06.009855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:06.016779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:06.048003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '518bcc4264ff4400a2a71e21d813a91c', 'request_id': 'req-9f187703-aa6f-46e0-9bc9-f274614d8e9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.069866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '518bcc4264ff4400a2a71e21d813a91c', 'request_id': 'req-9f187703-aa6f-46e0-9bc9-f274614d8e9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.084566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '518bcc4264ff4400a2a71e21d813a91c', 'request_id': 'req-9f187703-aa6f-46e0-9bc9-f274614d8e9a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.093263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 904/1820] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:06.094321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:06.110226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:06.112535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] The token cache is being invalidate because project 518bcc4264ff4400a2a71e21d813a91c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:06.113012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.113978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.117466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 917/1821] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:05 2022] DELETE /identity/v3/projects/518bcc4264ff4400a2a71e21d813a91c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:06.206947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7630c6ad-c6a7-4145-9696-11fdfbfd7cbb service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:06.517246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:06.532356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:06.532356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:06.532356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] PATH_INFO: `/v3/users/fa6c836585314c77a80459102cc1c84d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:06.532356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa6c836585314c77a80459102cc1c84d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:06.546114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:06.566232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:06.737629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa6c836585314c77a80459102cc1c84d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.777033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa6c836585314c77a80459102cc1c84d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.779494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa6c836585314c77a80459102cc1c84d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.779494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa6c836585314c77a80459102cc1c84d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.803672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:06.805398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:06.809007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5e1e0200-53b1-4772-bdc4-cbe780bf6217 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa6c836585314c77a80459102cc1c84d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:06.831679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f187703-aa6f-46e0-9bc9-f274614d8e9a tempest-VolumeQuotaClassesTest-2103794497 tempest-VolumeQuotaClassesTest-2103794497-project-admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:06.845467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:06.848473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:06.848473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:06.848473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] PATH_INFO: `/v3/users/a5ac1d9a881747de93aa411524e1015b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:06.850140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a5ac1d9a881747de93aa411524e1015b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:06.852558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 905/1822] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:06 2022] DELETE /identity/v3/users/fa6c836585314c77a80459102cc1c84d => generated 0 bytes in 759 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:06.855811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:06.857608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:06.858066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:06.859147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:06.905272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:06.912360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:07.016778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a5ac1d9a881747de93aa411524e1015b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.062210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a5ac1d9a881747de93aa411524e1015b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.063930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a5ac1d9a881747de93aa411524e1015b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.118035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a5ac1d9a881747de93aa411524e1015b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.118256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea27d93a-cd37-402f-be47-680ffae23c15 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a5ac1d9a881747de93aa411524e1015b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.174153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 918/1823] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:06 2022] DELETE /identity/v3/users/a5ac1d9a881747de93aa411524e1015b => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:07.175668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:07.175997 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:07.176301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:07.212770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:07.214053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:07.267464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 906/1824] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:07.268872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:07.270526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:07.313783 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3829a134-7dfd-45c1-a953-4e123ecdb5e6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:07.393899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 919/1825] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:07.400253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:07.402783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:07.456664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:07.545920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:07.547694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:07.549025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:07.550192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] PATH_INFO: `/v3/projects/75a4e048fae34fe898c5fea4f7e683cd` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:07.565969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=75a4e048fae34fe898c5fea4f7e683cd)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:07.569765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:07.749101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1e12e3-a63f-4c15-bc10-bc8d2a9df14d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:07.752262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:07.772036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:07.772748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:07.773032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:07.773312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:07.800302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75a4e048fae34fe898c5fea4f7e683cd', 'request_id': 'req-0749f685-2edc-4aad-9753-1a8f908032ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.813731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75a4e048fae34fe898c5fea4f7e683cd', 'request_id': 'req-0749f685-2edc-4aad-9753-1a8f908032ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.821704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:07.830728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:07.851132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75a4e048fae34fe898c5fea4f7e683cd', 'request_id': 'req-0749f685-2edc-4aad-9753-1a8f908032ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.897830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] The token cache is being invalidate because project 75a4e048fae34fe898c5fea4f7e683cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:07.897949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.907397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0749f685-2edc-4aad-9753-1a8f908032ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:07.911813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 920/1826] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 518 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:07.912732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:07.922038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 907/1827] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:07 2022] DELETE /identity/v3/projects/75a4e048fae34fe898c5fea4f7e683cd => generated 0 bytes in 650 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:07.927980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd200c-1cdb-4859-9e8d-bd44e2671301 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:07.928342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd200c-1cdb-4859-9e8d-bd44e2671301 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:07.928624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbd200c-1cdb-4859-9e8d-bd44e2671301 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:07.962524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed5779-4d19-442c-87e5-1e3bf32a8e59 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:08.034778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-adbd200c-1cdb-4859-9e8d-bd44e2671301 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:08.042556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-adbd200c-1cdb-4859-9e8d-bd44e2671301 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:08.158379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:08.159024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:08.159313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:08.159721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:08.167368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:08.295821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 908/1828] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:08.310519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:08.313797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:08.314781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:08.352275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:08.353728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:08.369817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:08.372086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:08.384409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 909/1829] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:08.389864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:08.398872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 921/1830] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 486 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:08.429573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503bb66-93ae-4207-b9de-3b774ed1919d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:08.439870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:08.440854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:08.441283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:08.441607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:08.442446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:08.443397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:08.447630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:08.508095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 910/1831] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:08.513253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:08.529841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e73f88d-d3bb-4c4a-975d-d6540bff8011 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:08.559326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:08.567132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:08.569586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:08.569974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:08.573932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:08.580243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:08.702806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 922/1832] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:08 2022] POST /identity/v3/projects => generated 363 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:08.708266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:08.733640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18efe602-1b12-4518-ba57-c3951ff5e030 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:09.037362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:09.038258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:09.039677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:09.045339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:09.046260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:09.048702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:09.155334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:09.168123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baef5af-b9f8-4f67-b2c2-878a6d234fda admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:09.170202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 911/1833] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:08 2022] POST /identity/v3/users => generated 282 bytes in 463 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:09.207757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:09.208457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:09.208764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:09.209021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:09.210117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:09.221426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:09.247565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 923/1834] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:09.249065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:09.268665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e8e279-5736-4851-b707-609405d6fa2e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:09.276403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:09.277228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:09.277500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:09.277781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:09.278778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:09.280239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b2683c-ce33-45d7-9e79-b3e951e253ac admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:09.310433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 912/1835] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:09 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:09.312534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:09.323635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cd32c0-1e74-4844-ad77-97714f1a788f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:09.343835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:09.344579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:09.344864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:09.345947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] PATH_INFO: `/v3/projects/fcbe81af8d8c4de9b8a76b982d1c0b4a/users/d129bf277edf454386c0c3e679fd7c63/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:09.409705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcbe81af8d8c4de9b8a76b982d1c0b4a, user_id=d129bf277edf454386c0c3e679fd7c63, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:09.413111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:09.449987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 924/1836] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:09 2022] PUT /identity/v3/projects/fcbe81af8d8c4de9b8a76b982d1c0b4a/users/d129bf277edf454386c0c3e679fd7c63/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:09.452399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:09.452693 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:09.453030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:09.548497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:09.572733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] MFA Rules not processed for user `d129bf277edf454386c0c3e679fd7c63`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:09.578668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:09.783735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 913/1837] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:09 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:09.788888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:09.795562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:10.016892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b11d5a5-67d9-4942-90a0-80890aaf3d36 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:10.040218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:10.044370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:10.045001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:10.045482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:10.053375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:10.055747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:10.159028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 925/1838] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:09 2022] POST /identity/v3/projects => generated 363 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:10.163400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:10.184810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b02650-4e7d-4813-885c-e0cc1edccc7c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:10.419989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:10.420744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:10.421039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:10.421514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:10.422143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:10.428674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:10.491705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 914/1839] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:10 2022] POST /identity/v3/users => generated 288 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:10.492954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:10.513606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e8a4b8-5bc8-4e4b-a750-2f1ad7dd39d3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:10.548608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:10.549228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:10.549491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:10.549945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:10.550649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:10.555793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:10.608804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 926/1840] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:10.609546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:10.624460 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86d975-a8ac-4e2b-bfb3-bb1f91c83419 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:10.651185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:10.652002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:10.652350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:10.652636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] PATH_INFO: `/v3/projects/7a324515436b402097b99949c2497e42/users/ab37262675ab43e9ad17696d35567f57/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:10.696663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a324515436b402097b99949c2497e42, user_id=ab37262675ab43e9ad17696d35567f57, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:10.708144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:10.765535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 915/1841] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:10 2022] PUT /identity/v3/projects/7a324515436b402097b99949c2497e42/users/ab37262675ab43e9ad17696d35567f57/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:10.766973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:10.778387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a505fdf7-2c53-43b8-a6b5-c88f542e74d5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:10.991857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:10.993484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:10.993968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:10.994635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:10.995443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:11.002914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237067e3-1564-4084-8ed7-edc3f3c74a91 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:11.063603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:11.063926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 927/1842] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:10 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:11.082599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780fa95-d5f6-4204-9ec7-816ac6fa28a2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:11.094288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:11.098596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:11.099770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:11.100272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] PATH_INFO: `/v3/projects/7a324515436b402097b99949c2497e42/users/ab37262675ab43e9ad17696d35567f57/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:11.109279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a324515436b402097b99949c2497e42, user_id=ab37262675ab43e9ad17696d35567f57, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:11.112490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:11.134023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 916/1843] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:11 2022] PUT /identity/v3/projects/7a324515436b402097b99949c2497e42/users/ab37262675ab43e9ad17696d35567f57/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:11.147631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:11.147950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:11.148235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:11.213772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] MFA Rules not processed for user `ab37262675ab43e9ad17696d35567f57`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:11.215164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:11.570502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:11.639446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 928/1844] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:11 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 494 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:11.658618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:11.660490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:11.735673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:11.740610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:11.925057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d4fff4-c4c8-44a8-81c8-0d941a5e69d3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:11.938047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:11.938704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:11.938989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:11.939266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:11.959520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:11.960994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:11.975187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:12.006963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 917/1845] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:11 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:12.008327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:12.012489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.017147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65dea79e-6d51-48a8-82b7-2f3b6e7a864b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:12.043657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:12.043657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.043657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.043657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] PATH_INFO: `/v3/projects/49112a7a3839497aa5a24e26f6410387` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.044536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49112a7a3839497aa5a24e26f6410387)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:12.058602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:12.071466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c8837e-7816-4f46-838d-2cdc2bae39d2 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:12.106922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:12.110054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.110280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.110611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] PATH_INFO: `/v3/projects/d739e0e346a1442f80113f14e1b4a603` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.121496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d739e0e346a1442f80113f14e1b4a603)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:12.128079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:12.223119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49112a7a3839497aa5a24e26f6410387', 'request_id': 'req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.255963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49112a7a3839497aa5a24e26f6410387', 'request_id': 'req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.282255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49112a7a3839497aa5a24e26f6410387', 'request_id': 'req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.303360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd739e0e346a1442f80113f14e1b4a603', 'request_id': 'req-c5888ce5-4203-42f7-80d6-9815f12fe585'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.313884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd739e0e346a1442f80113f14e1b4a603', 'request_id': 'req-c5888ce5-4203-42f7-80d6-9815f12fe585'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.331433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] The token cache is being invalidate because project 49112a7a3839497aa5a24e26f6410387 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:12.331795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.336207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7524d1d3-c0e9-47b7-86de-4a84e7ae4fa4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.337085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd739e0e346a1442f80113f14e1b4a603', 'request_id': 'req-c5888ce5-4203-42f7-80d6-9815f12fe585'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.338269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 929/1846] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:11 2022] DELETE /identity/v3/projects/49112a7a3839497aa5a24e26f6410387 => generated 0 bytes in 603 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:12.339758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.340211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.340636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.368513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] The token cache is being invalidate because project d739e0e346a1442f80113f14e1b4a603 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:12.368812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.376513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:12.387275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 918/1847] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:12 2022] DELETE /identity/v3/projects/d739e0e346a1442f80113f14e1b4a603 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:12.388414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:12.391753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.436250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] MFA Rules not processed for user `95e33684ef7f4712a9dc517b28d29071`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:12.442175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.608576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5888ce5-4203-42f7-80d6-9815f12fe585 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:12.620138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:12.620809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.621096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.621380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.632474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 930/1848] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:12 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:12.632474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:12.633373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.634154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.704578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:12.788649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:12.792598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:12.815650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 919/1849] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:12 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:12.817827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:12.822179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:12.825809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58f155-927f-4a1f-ad78-a8f7afc66019 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:12.840322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:12.842442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.843666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.844801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] PATH_INFO: `/v3/users/7399f5d53c4c4c40baa14e919d5accd0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.848778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7399f5d53c4c4c40baa14e919d5accd0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:12.852583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220dbbe7-f28c-4264-bf73-7a55d8e63767 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:12.854342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:12.867801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:12.868904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:12.871950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:12.872265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] PATH_INFO: `/v3/users/426e7bf4d5014becb8e044d3a7e158c5` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:12.876623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=426e7bf4d5014becb8e044d3a7e158c5)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:12.883127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:12.966101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:12.996907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '426e7bf4d5014becb8e044d3a7e158c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.019654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7399f5d53c4c4c40baa14e919d5accd0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.033507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '426e7bf4d5014becb8e044d3a7e158c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.036174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '426e7bf4d5014becb8e044d3a7e158c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.043111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '426e7bf4d5014becb8e044d3a7e158c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.074995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '426e7bf4d5014becb8e044d3a7e158c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.076827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7399f5d53c4c4c40baa14e919d5accd0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.079923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7399f5d53c4c4c40baa14e919d5accd0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.114462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7399f5d53c4c4c40baa14e919d5accd0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.119214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7399f5d53c4c4c40baa14e919d5accd0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:13.125303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 920/1850] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:12 2022] DELETE /identity/v3/users/426e7bf4d5014becb8e044d3a7e158c5 => generated 0 bytes in 308 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:13.127254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:13.130677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:13.188127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 931/1851] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:12 2022] DELETE /identity/v3/users/7399f5d53c4c4c40baa14e919d5accd0 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:13.408995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2bd5c9-3910-4fd8-87d9-d717821675bb admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:13.624873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:13.625573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:13.625907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:13.626185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:13.626965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:13.640033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:13.729289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 921/1852] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:13 2022] POST /identity/v3/projects => generated 371 bytes in 605 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:13.733889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:13.735812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:13.780543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcafaf10-ab0c-4a70-be09-535ae64e3d00 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:13.894591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:13.896323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:13.896836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:13.897325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:13.898229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:13.900597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:13.947168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 932/1853] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:13 2022] POST /identity/v3/users => generated 292 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:13.951588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:13.983532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f9872-bc9d-41d5-9416-82d6799316b0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:13.999392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:14.000378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.000916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.001281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.002275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:14.004272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:14.039626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 922/1854] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:14.041113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:14.065502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda41cff-8d10-4bbf-b908-53e2c8e058c9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:14.075454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:14.076332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.076728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.077279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] PATH_INFO: `/v3/projects/2f925e2803b9448397a37c7b053651b8/users/4b47dcf1992142609e08f7aaebd9519e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.137762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f925e2803b9448397a37c7b053651b8, user_id=4b47dcf1992142609e08f7aaebd9519e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:14.162649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:14.202018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 933/1855] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:14 2022] PUT /identity/v3/projects/2f925e2803b9448397a37c7b053651b8/users/4b47dcf1992142609e08f7aaebd9519e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:14.206628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:14.230420 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380164ab-406a-43f1-a78f-22796dd7628d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:14.298472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:14.384343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:14.385161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.385560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.385953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.386787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:14.388551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d1d9e-0fcd-4b1f-9ae5-737452256f8f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:14.419469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 923/1856] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:14 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:14.423806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:14.437741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:14.450421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166efefd-0200-4871-99d8-ac2fdc2e950f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:14.457460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:14.459774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.460411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.462611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] PATH_INFO: `/v3/projects/2f925e2803b9448397a37c7b053651b8/users/4b47dcf1992142609e08f7aaebd9519e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.466066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2f925e2803b9448397a37c7b053651b8, user_id=4b47dcf1992142609e08f7aaebd9519e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:14.480301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592239b7-51db-4f3d-b467-dda59f871753 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:14.508557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 934/1857] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:14 2022] PUT /identity/v3/projects/2f925e2803b9448397a37c7b053651b8/users/4b47dcf1992142609e08f7aaebd9519e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:14.559244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.559562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.559846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.561925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 924/1858] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:23:14.677681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:14.678158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:14.678598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:14.742412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:14.745985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:14.944827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 935/1859] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:23:14 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:15.020637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:15.023038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:15.061898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddcbe3-5b32-4124-8520-a4ce419e5f33 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:15.080565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:15.082643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:15.086065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:15.086624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:15.116850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:15.123724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557a64c7-0223-4075-8460-c6471a8e8eb1 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:15.160245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 925/1860] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:15.995534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:15.997440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:16.226095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad794c0-6ab3-417e-a889-6387abfad2ac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:16.235738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:16.238010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:16.238408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:16.240206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] PATH_INFO: `/v3/projects/92d8851648184bca808adc9d8b3dfcc8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:16.245584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] RBAC: Authorizing `identity:delete_project(project_id=92d8851648184bca808adc9d8b3dfcc8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:16.248671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:16.438249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '92d8851648184bca808adc9d8b3dfcc8', 'request_id': 'req-596e8723-ac9d-4ee5-9e3d-ce097eb64360'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:16.454603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '92d8851648184bca808adc9d8b3dfcc8', 'request_id': 'req-596e8723-ac9d-4ee5-9e3d-ce097eb64360'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:16.474482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '92d8851648184bca808adc9d8b3dfcc8', 'request_id': 'req-596e8723-ac9d-4ee5-9e3d-ce097eb64360'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:16.494750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] The token cache is being invalidate because project 92d8851648184bca808adc9d8b3dfcc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:16.495454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:16.496346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:16.500513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 936/1861] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:15 2022] DELETE /identity/v3/projects/92d8851648184bca808adc9d8b3dfcc8 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:16.505758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dab21-cd60-44f0-ba5b-a6e2c75816e6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:16.506314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dab21-cd60-44f0-ba5b-a6e2c75816e6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:16.506392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41dab21-cd60-44f0-ba5b-a6e2c75816e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:16.600614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e41dab21-cd60-44f0-ba5b-a6e2c75816e6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:16.607813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e41dab21-cd60-44f0-ba5b-a6e2c75816e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:16.738858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:16.748046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:16.789411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:16.814184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 926/1862] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 309 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:16.817625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:16.817921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:16.818240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:16.896441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:16.900350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:16.930097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 927/1863] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:16.933024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:16.962381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12719f65-10b1-4e92-932a-16186e37dfef None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:16.975774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:16.978475 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:16.979886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:16.980914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:16.982087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:16.984235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:16.991474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:16.994633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596e8723-ac9d-4ee5-9e3d-ce097eb64360 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.017833 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.018510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.018796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.019075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] PATH_INFO: `/v3/projects/bd52358d165a4431aff712c4ee62317a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.028628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd52358d165a4431aff712c4ee62317a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.030665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.038530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 928/1864] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:17.040207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.072400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3365828b-1326-49b4-8577-11b4582b0d75 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.102703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.103501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.103720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.108858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.111930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.120706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.227039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd52358d165a4431aff712c4ee62317a', 'request_id': 'req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:17.243261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd52358d165a4431aff712c4ee62317a', 'request_id': 'req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:17.263852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 929/1865] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:17.268407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.275821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd52358d165a4431aff712c4ee62317a', 'request_id': 'req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:17.307172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] The token cache is being invalidate because project bd52358d165a4431aff712c4ee62317a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:17.307496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:17.310435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3d7602fb-49e8-4f3f-9d6a-c4b43e186c9e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:17.320034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 937/1866] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:16 2022] DELETE /identity/v3/projects/bd52358d165a4431aff712c4ee62317a => generated 0 bytes in 580 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:17.322045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fa711-25b9-4a8e-9453-905420399d5f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.322774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fa711-25b9-4a8e-9453-905420399d5f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.322774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31fa711-25b9-4a8e-9453-905420399d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.352830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b31fa711-25b9-4a8e-9453-905420399d5f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:17.354085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b31fa711-25b9-4a8e-9453-905420399d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:17.407937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fb6e88-6803-4820-ad46-909ad90f42df admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.416986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.418721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.419511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.420151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.421849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.425200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.452270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:17.467312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 930/1867] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:17.471135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.475225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:17.499284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 938/1868] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:17.500564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abca616-d1c5-4a70-ae92-e09f35faf79b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.500777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.501071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.501353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.515789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.517801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.518436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.519697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.520714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.528455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.542006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:17.545021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:17.549725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 931/1869] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:17.551889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.568330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 939/1870] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:17.569053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1801dd-2ecd-483a-8f6d-2b3c1968cc08 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.569328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.591867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.594724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.596090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.596701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.599549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.602992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.607498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc6e371-4ec7-4d60-9bee-603238fc0f9b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.622834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.623458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.623766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.624043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.626688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:17.626688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.629018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 932/1871] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:17.629655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.637076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.663641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 940/1872] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:17.667253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.690105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f203ff-1901-4b26-b239-aaedea38b057 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.703593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.704208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.704542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.704868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.705548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.707904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.709545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9791d952-6321-458f-9fc9-47843840038c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:17.724432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:17.724978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.725547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.728124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] PATH_INFO: `/v3/projects/9c079945b0bd4cfb8f4e1b2199057507/users/5c76940c08e946cdb7c15806b7b30cb7/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.771694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c079945b0bd4cfb8f4e1b2199057507, user_id=5c76940c08e946cdb7c15806b7b30cb7, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:17.772548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 941/1873] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:17.776459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29562fb0-8a2d-4427-8266-e66f52be852c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:17.777588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:17.791867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:17.823063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 933/1874] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:17 2022] PUT /identity/v3/projects/9c079945b0bd4cfb8f4e1b2199057507/users/5c76940c08e946cdb7c15806b7b30cb7/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:17.842065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:17.842357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:17.842637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:17.901744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] MFA Rules not processed for user `5c76940c08e946cdb7c15806b7b30cb7`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:17.908098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:17.919916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0559e314-1829-4e13-98c9-68791a3762a4 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.108517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.109309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.109613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.109875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.110692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.112557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.126250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 934/1875] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:18.132532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.160498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57123c36-6661-4ef3-9d17-7ccf19bbd928 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.176444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.177291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.177522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.177799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.178758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 942/1876] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:17 2022] POST /identity/v3/users => generated 304 bytes in 402 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:18.179516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.181601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.183002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.203352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afae00f-ef00-4346-bad0-90a065abf21d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.215758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.216395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.216672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.216947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.217650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.219299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.239946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 943/1877] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:18.245775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.250582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 935/1878] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:18 2022] POST /identity/v3/projects => generated 375 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:18.251872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.281549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709de820-38ae-441e-9341-5a16dc5cf459 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.439951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.441519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.441803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.442078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.443785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.445436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.462638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 944/1879] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:18.466931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.480554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea068a4-cd64-40c1-b916-a9ae13e59451 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.482320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a3468-9ef5-41d4-84e7-bc3f5d71c194 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.491599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.492435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.492770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.493534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] PATH_INFO: `/v3/projects/999cecea09824837a577066ed19abea3/users/3d2ae20b39d64633a64c007f19fd3fb3/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.512246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.521086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.523411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.524030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.526508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.531534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] RBAC: Authorizing `identity:create_grant(project_id=999cecea09824837a577066ed19abea3, user_id=3d2ae20b39d64633a64c007f19fd3fb3, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.534669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.535609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e16cfe27-8d51-4035-966c-23b95f79529e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.555227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:18.581192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 945/1880] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:18 2022] PUT /identity/v3/projects/999cecea09824837a577066ed19abea3/users/3d2ae20b39d64633a64c007f19fd3fb3/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:18.585819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.586404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.589045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.609644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 936/1881] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:18 2022] POST /identity/v3/users => generated 294 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:18.614056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.637963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b082f77b-615b-4049-ae9f-bbe5f4b3ac2f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.650346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:18.651764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] MFA Rules not processed for user `3d2ae20b39d64633a64c007f19fd3fb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:18.652821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:18.748296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.749351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.749850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.750350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.751279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.753198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.773572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 946/1882] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:18 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:18.776040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.779123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 937/1883] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:18.782981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.793968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9201ba22-5b3a-414d-9441-253601563414 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.799995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.801277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.801277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.801933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] PATH_INFO: `/v3/projects/902d46a9eb05471aa2e6d12eb1976a96/users/9afe39956ac849068c6f036adf15716d/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.822122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] RBAC: Authorizing `identity:create_grant(project_id=902d46a9eb05471aa2e6d12eb1976a96, user_id=9afe39956ac849068c6f036adf15716d, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.825337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.841990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85b4224-72cd-45b8-bb8d-444b96e24ce3 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:18.851587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 938/1884] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:18 2022] PUT /identity/v3/projects/902d46a9eb05471aa2e6d12eb1976a96/users/9afe39956ac849068c6f036adf15716d/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:18.855814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:18.859626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:18.861218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:18.861591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:18.861872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:18.862556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:18.873258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:18.962871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 947/1885] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:18 2022] POST /identity/v3/projects => generated 409 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:18.967596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.021633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df3fce-273f-4b35-8296-019db105729c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.063793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99df30a-07d5-4ca5-9405-dfdebbb47e8d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.070321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.071403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.072287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.072725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.074164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.077427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.110780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 948/1886] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:18 2022] POST /identity/v3/users => generated 311 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:19.114314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.117642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.118477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.118760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.119080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.119851 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.121503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.129747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651ba9c-961a-4f82-80e3-7a9fb22f41ca admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.135453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.136319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.136723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.137107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.138210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.140376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.145364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 939/1887] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:18 2022] GET /identity/v3/roles => generated 1420 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:19.151731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.155239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 949/1888] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:19.159518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.163196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4dee2-7edd-4bf3-9ac9-1733550d1048 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.167938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.168610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.168888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.169163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] PATH_INFO: `/v3/projects/902d46a9eb05471aa2e6d12eb1976a96/users/9afe39956ac849068c6f036adf15716d/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.170251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffa6f22-3688-403a-9708-6fd351876c34 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.171172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=902d46a9eb05471aa2e6d12eb1976a96, user_id=9afe39956ac849068c6f036adf15716d, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.173874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931e0042-5c42-4925-85f3-1a82c44b9ac7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.175269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.176098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.176459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.176987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] PATH_INFO: `/v3/projects/3408ac4c39464c57a526e22e3047eb02/users/dc589d6cbc414b318f300f36c351b09e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.185232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 940/1889] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:19 2022] PUT /identity/v3/projects/902d46a9eb05471aa2e6d12eb1976a96/users/9afe39956ac849068c6f036adf15716d/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:19.196944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.197450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.197738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.202181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3408ac4c39464c57a526e22e3047eb02, user_id=dc589d6cbc414b318f300f36c351b09e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.204984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.223730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 950/1890] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:19 2022] PUT /identity/v3/projects/3408ac4c39464c57a526e22e3047eb02/users/dc589d6cbc414b318f300f36c351b09e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:19.229382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.230890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:19.232779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] MFA Rules not processed for user `9afe39956ac849068c6f036adf15716d`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:19.234612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.259651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:19.305146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66990a94-2701-4601-9b5a-d8e46090cdf1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.313449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.314112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.314400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.314707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.315456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.320788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.333852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 951/1891] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:19.338110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.344634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 941/1892] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:19 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:19.351337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3da2e7-fd07-4b92-9bdb-933a87ccbed1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.353031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.355012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.360812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.361761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.365723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.366081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] PATH_INFO: `/v3/projects/3408ac4c39464c57a526e22e3047eb02/users/dc589d6cbc414b318f300f36c351b09e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.368506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3408ac4c39464c57a526e22e3047eb02, user_id=dc589d6cbc414b318f300f36c351b09e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.371306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b7986-37ac-4128-bc75-5753f1eda9f6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.415526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 952/1893] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:19 2022] PUT /identity/v3/projects/3408ac4c39464c57a526e22e3047eb02/users/dc589d6cbc414b318f300f36c351b09e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:19.419350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845ef0-0b7e-466f-b0ae-65a2e111089c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.419716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845ef0-0b7e-466f-b0ae-65a2e111089c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.419995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41845ef0-0b7e-466f-b0ae-65a2e111089c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.455066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8d1b91-0ad2-4f87-8bda-adb0dad412f0 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.456423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-41845ef0-0b7e-466f-b0ae-65a2e111089c None None] MFA Rules not processed for user `dc589d6cbc414b318f300f36c351b09e`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:19.457622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-41845ef0-0b7e-466f-b0ae-65a2e111089c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.516682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.517328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.517607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.517880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.547399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 953/1894] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:19 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:19.553964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.554447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.554550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.556715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 954/1895] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:23:19 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:23:19.563226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.565989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.584693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc42601-a4b7-4f55-8a8b-6b2065e22f4b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.589094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.589731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.590012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.590392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.603297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.605415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a78a2cb-91e4-4dba-bd2a-35a7e69b11ea service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.619080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 955/1896] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:19.620346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.620862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.621154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.621496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.621869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e9aa2e8-d2ed-4930-901e-6b24e4584410 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.641380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 942/1897] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:19.643992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.644712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.644712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.652810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:19.654828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.672036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:19.689465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] MFA Rules not processed for user `4b47dcf1992142609e08f7aaebd9519e`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:19.690724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.761920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 956/1898] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:23:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:19.775159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 943/1899] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:19 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:19.780401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.782074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.794481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:19.811361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57354cce-8b1a-4b92-9600-513332a0e11f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.817298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.817984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.818268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.818548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.819375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea7cf98-2a89-4f4b-9c4d-c360f7bce202 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:19.824259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:19.825155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:19.825435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:19.825713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:19.827690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:19.828530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:19.830187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.832121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.854309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 957/1900] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:19.921567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:19.923189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:19.935920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 944/1901] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:19 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:20.319438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:20.345106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8acb88-d81a-485f-a6a7-f6428f3e09b4 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:20.349316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:20.349993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:20.350812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:20.350812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:20.429298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:20.430955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:20.440938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 958/1902] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:20 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:22.524580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:22.533461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91de035-d8e1-4d9c-b304-fccdf81cb6f6 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:22.537806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:22.538542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:22.538688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:22.539577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:22.549119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:22.550541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e7e3e-9868-4cb8-997b-bd148b7a9e19 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:22.585518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 945/1903] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:22 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:23.820085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:23.903591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1800906e-56ac-40d2-aebb-0e34cd4706a5 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:23.909159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:23.909739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:23.910027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:23.910395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] PATH_INFO: `/v3/users/5c76940c08e946cdb7c15806b7b30cb7` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:23.912379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c76940c08e946cdb7c15806b7b30cb7)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:23.914377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:23.980917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c76940c08e946cdb7c15806b7b30cb7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:23.995971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c76940c08e946cdb7c15806b7b30cb7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:23.996859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c76940c08e946cdb7c15806b7b30cb7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.007828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c76940c08e946cdb7c15806b7b30cb7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.008591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c76940c08e946cdb7c15806b7b30cb7'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.038438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 959/1904] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:23 2022] DELETE /identity/v3/users/5c76940c08e946cdb7c15806b7b30cb7 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:24.038586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:24.038586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:24.038586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:24.056622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:24.057861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:24.149166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 946/1905] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:24.160284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:24.162548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:24.179316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:24.244378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73117143-8839-47bf-b5d0-48e1937c91f1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:24.248590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:24.249793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:24.249952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:24.250268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:24.262164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:24.264742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:24.281235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 960/1906] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:24.589477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:24.590971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:24.602894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93210005-584c-4dea-ac7e-4b53897399a1 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:24.607469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:24.608119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:24.608392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:24.608659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] PATH_INFO: `/v3/projects/9c079945b0bd4cfb8f4e1b2199057507` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:24.609971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c079945b0bd4cfb8f4e1b2199057507)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:24.611909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:24.627954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:24.659103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c079945b0bd4cfb8f4e1b2199057507', 'request_id': 'req-1e9e950c-d5ca-43a9-9519-9590f0c9836c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.665972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c079945b0bd4cfb8f4e1b2199057507', 'request_id': 'req-1e9e950c-d5ca-43a9-9519-9590f0c9836c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.673582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c079945b0bd4cfb8f4e1b2199057507', 'request_id': 'req-1e9e950c-d5ca-43a9-9519-9590f0c9836c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.684354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] The token cache is being invalidate because project 9c079945b0bd4cfb8f4e1b2199057507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:24.684641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.685265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.686768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 947/1907] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:24 2022] DELETE /identity/v3/projects/9c079945b0bd4cfb8f4e1b2199057507 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:24.690789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:24.700540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453f9481-89f9-4281-8ab0-125cf23ad547 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:24.755095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:24.755774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:24.756054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:24.756329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] PATH_INFO: `/v3/users/9afe39956ac849068c6f036adf15716d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:24.757495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9afe39956ac849068c6f036adf15716d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:24.759474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:24.803880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9afe39956ac849068c6f036adf15716d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.817597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9afe39956ac849068c6f036adf15716d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.818280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9afe39956ac849068c6f036adf15716d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.826990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9afe39956ac849068c6f036adf15716d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.827670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-158e203d-78ac-411d-8446-dbf313fdc6f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9afe39956ac849068c6f036adf15716d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:24.839861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 961/1908] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:24 2022] DELETE /identity/v3/users/9afe39956ac849068c6f036adf15716d => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:25.209024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.210621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:25.227537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9e950c-d5ca-43a9-9519-9590f0c9836c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.288072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.290758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.290758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.290758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] PATH_INFO: `/v3/projects/902d46a9eb05471aa2e6d12eb1976a96` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.290758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=902d46a9eb05471aa2e6d12eb1976a96)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.292725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.345286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '902d46a9eb05471aa2e6d12eb1976a96', 'request_id': 'req-ea763cc9-b3bc-4ca0-9b23-2aba18799336'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:25.353914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '902d46a9eb05471aa2e6d12eb1976a96', 'request_id': 'req-ea763cc9-b3bc-4ca0-9b23-2aba18799336'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:25.364898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '902d46a9eb05471aa2e6d12eb1976a96', 'request_id': 'req-ea763cc9-b3bc-4ca0-9b23-2aba18799336'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:25.380255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] The token cache is being invalidate because project 902d46a9eb05471aa2e6d12eb1976a96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:25.380476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:25.381140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ea763cc9-b3bc-4ca0-9b23-2aba18799336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:25.382918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 948/1909] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:25 2022] DELETE /identity/v3/projects/902d46a9eb05471aa2e6d12eb1976a96 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:25.388370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.388953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.389247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.414038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:25.415230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:25.445554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:25.477982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 962/1910] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:25.485517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.485804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.486083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.506479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:25.507709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:25.515778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 949/1911] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:25.520138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.530632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c154bc1-9f68-41b2-b0e0-4479d12409e6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.535239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.535925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.536334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.536504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.537201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:25.537508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.539260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.550928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 963/1912] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:25.555240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.564332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da2e964-d74a-42c2-bb62-52b8438b5454 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.569633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.570251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.570538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.570812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.571522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.573397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.602852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 950/1913] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/projects => generated 385 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:25.607037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.616878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d57b3e1-dc53-43a3-8bee-c4200cc354d5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.673770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.674316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.674605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.674889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.675598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.677694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.702198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 964/1914] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:25.706328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.713667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:25.717948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c94222-956f-4046-9aa0-4b8f9a94beeb admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.724298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.725205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.725529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.726168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.727146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.729601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.740571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 951/1915] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:25.744687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.754634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa13597-d5a5-40ea-9d27-d641d823a114 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.759853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.760463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.760746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.761060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.761738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.763380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb18be66-6332-4385-a67f-5c0653418914 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.774439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 965/1916] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:25.778524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.788284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f4888-b35b-44b1-8cfb-1b2868a47d8a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:25.792847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:25.793511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.793795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.794074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] PATH_INFO: `/v3/projects/03f653b370f24cea916a4dac5cf8555b/users/03043fdf8fc943fe8334c8aa6052d298/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.808882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] RBAC: Authorizing `identity:create_grant(project_id=03f653b370f24cea916a4dac5cf8555b, user_id=03043fdf8fc943fe8334c8aa6052d298, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:25.811548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:25.824390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 952/1917] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:25 2022] PUT /identity/v3/projects/03f653b370f24cea916a4dac5cf8555b/users/03043fdf8fc943fe8334c8aa6052d298/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:25.834862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:25.835316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:25.835403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:25.862301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] MFA Rules not processed for user `03043fdf8fc943fe8334c8aa6052d298`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:25.863545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:25.932453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 966/1918] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:25.937078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:25.997935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840cac9-808c-4134-9ef9-1d6dfda8548b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.002665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.003292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.003615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.003895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.004577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.007105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.034629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 953/1919] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:25 2022] POST /identity/v3/projects => generated 381 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:26.038936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.051362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9e112-34c2-486c-9da2-532cc90e280b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.085187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:26.117767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.118510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.118859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.119207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.119990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.122155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.151763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 967/1920] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:26 2022] POST /identity/v3/users => generated 291 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:26.155546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.166813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3981dcf-b416-4a9b-9af6-047bfeb958d7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.171847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.172498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.172789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.173066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.173780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.175371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.187086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 954/1921] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:26.191808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.204445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc38d3-c155-4a45-9084-508bc7816e3f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.210064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.210772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.211126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.211507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.212315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.213983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd01ff23-ef99-4de9-a8af-44fc5e87bbde admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.227621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 968/1922] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:26.229182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.239995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b99b862-d3af-4452-afad-388889d1cd68 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.245961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.246755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.247134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.247541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] PATH_INFO: `/v3/projects/039ba2ed685f4b9ea69861ecd62e087e/users/8968abe08d3441e2960f2bae06d7682d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.263340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=039ba2ed685f4b9ea69861ecd62e087e, user_id=8968abe08d3441e2960f2bae06d7682d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.266133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.284295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 955/1923] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:26 2022] PUT /identity/v3/projects/039ba2ed685f4b9ea69861ecd62e087e/users/8968abe08d3441e2960f2bae06d7682d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:26.295749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.296185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.296568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.351892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] MFA Rules not processed for user `8968abe08d3441e2960f2bae06d7682d`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:26.353422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:26.461972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 969/1924] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:26 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:26.467003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.595875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502891b6-f278-4a6c-b51c-c35c058baba6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.605959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.606776 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.607770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.608058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.608900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.611003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.666448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 956/1925] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:26 2022] POST /identity/v3/projects => generated 385 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:26.672389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.715232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d73934-f2bd-478b-b031-a5988d09bc64 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.834115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.835509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.836401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.837174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.838005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.841019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.868322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 970/1926] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:26 2022] POST /identity/v3/users => generated 299 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:26.873198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.880628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:26.884452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29cbf29-ae3b-4ffc-bf0c-58170c5eff91 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.891229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.891885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.892164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.892503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.893167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.895113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.906628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 957/1927] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:26.910867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.921804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295003b-b223-411f-87de-2e529dd09541 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:26.927619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:26.928355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:26.928808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:26.929182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] PATH_INFO: `/v3/projects/55855b449c8e4f3f8de99f39719f9837/users/1d06405c59f84572b47a7f3ee4ac816e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:26.945326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55855b449c8e4f3f8de99f39719f9837, user_id=1d06405c59f84572b47a7f3ee4ac816e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:26.948177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:26.962158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 971/1928] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:26 2022] PUT /identity/v3/projects/55855b449c8e4f3f8de99f39719f9837/users/1d06405c59f84572b47a7f3ee4ac816e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:26.966502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:26.976881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd80fe14-a7c2-4c05-9e04-a926966fda40 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:27.032887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:27.033514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:27.033794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:27.034068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:27.034761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:27.036965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a43fecf-dcde-4f59-a029-f45d83eff6c6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:27.046802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 958/1929] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:26 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:27.052485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:27.059523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:27.063480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad574e11-8d91-4b1a-b0d3-1b770069d53e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:27.069809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:27.070462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:27.070742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:27.071018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] PATH_INFO: `/v3/projects/55855b449c8e4f3f8de99f39719f9837/users/1d06405c59f84572b47a7f3ee4ac816e/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:27.073048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] RBAC: Authorizing `identity:create_grant(project_id=55855b449c8e4f3f8de99f39719f9837, user_id=1d06405c59f84572b47a7f3ee4ac816e, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:27.075998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:27.086776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 972/1930] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:27 2022] PUT /identity/v3/projects/55855b449c8e4f3f8de99f39719f9837/users/1d06405c59f84572b47a7f3ee4ac816e/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:27.096894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:27.097185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:27.097466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:27.124319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] MFA Rules not processed for user `1d06405c59f84572b47a7f3ee4ac816e`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:27.125517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:27.194029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 959/1931] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:27 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:27.205359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:27.207072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:27.268494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2423bb-cbd9-46e6-963c-cf0ab752c852 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:27.276592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:27.276592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:27.276592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:27.276592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:27.347960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:27.349754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945d67b8-1574-4eda-8918-32888eee1072 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:27.361431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 973/1932] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:27 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:28.138143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:28.139580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:28.209638 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c44b52-a782-41a7-bb85-414c3fc1c72e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:28.214352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:28.215018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:28.215377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:28.215780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] PATH_INFO: `/v3/users/d129bf277edf454386c0c3e679fd7c63` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:28.217058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d129bf277edf454386c0c3e679fd7c63)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:28.219074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:28.235302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:28.266722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd129bf277edf454386c0c3e679fd7c63'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:28.278573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd129bf277edf454386c0c3e679fd7c63'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:28.279244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd129bf277edf454386c0c3e679fd7c63'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:28.279886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd129bf277edf454386c0c3e679fd7c63'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:28.286747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd129bf277edf454386c0c3e679fd7c63'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:28.299249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 960/1933] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:28 2022] DELETE /identity/v3/users/d129bf277edf454386c0c3e679fd7c63 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:28.304432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:28.304928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:28.305021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:28.323949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:28.325389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:28.385785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 974/1934] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:28.396005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:28.397417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:28.463450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5339986-6a55-40e5-a950-13af7644b5f4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:28.469317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:28.469953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:28.470427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:28.470578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:28.487081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:28.488807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:28.505257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 961/1935] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:29.070620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:29.072214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:29.152745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:29.173162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37559115-591a-4a98-ad72-16dadb594d3c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:29.180332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:29.181095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:29.181255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:29.181534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:29.187131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:29.300382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:29.301897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f0fad44-a8ac-4da9-b245-e68710492b54 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:29.327150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 975/1936] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:29 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 257 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:29.700574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:29.701032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:29.722524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d775cd59-66f7-4323-bb9a-cc1a6ed9a871 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:29.727805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:29.728662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:29.729139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:29.729562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] PATH_INFO: `/v3/projects/fcbe81af8d8c4de9b8a76b982d1c0b4a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:29.730883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcbe81af8d8c4de9b8a76b982d1c0b4a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:29.733054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:29.760059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c0ea-f49b-4234-b745-1ff087c376ba None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:29.760585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c0ea-f49b-4234-b745-1ff087c376ba None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:29.761035 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889c0ea-f49b-4234-b745-1ff087c376ba None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:29.763318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 976/1937] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:23:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:23:29.792189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:29.792673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:29.793120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:29.821375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcbe81af8d8c4de9b8a76b982d1c0b4a', 'request_id': 'req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:29.837180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcbe81af8d8c4de9b8a76b982d1c0b4a', 'request_id': 'req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:29.839995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:29.843939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:29.855300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcbe81af8d8c4de9b8a76b982d1c0b4a', 'request_id': 'req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:29.884712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] The token cache is being invalidate because project fcbe81af8d8c4de9b8a76b982d1c0b4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:29.885250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:29.887002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:29.890166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 962/1938] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:29 2022] DELETE /identity/v3/projects/fcbe81af8d8c4de9b8a76b982d1c0b4a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:29.894572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:29.896732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:29.946343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:30.027612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 977/1939] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:23:29 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:30.040265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6f84b9-1ece-4d42-a2ed-5079fd983b8e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:30.045798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:30.046516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.046802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.047081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] PATH_INFO: `/v3/users/ab37262675ab43e9ad17696d35567f57` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.048676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ab37262675ab43e9ad17696d35567f57)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:30.051146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:30.091840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:30.093784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.112134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5421c8b6-bbe3-4f92-967d-c2413e6f1c1c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:30.127707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:30.127707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.127707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.127707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.134782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:30.136628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:30.137795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ab37262675ab43e9ad17696d35567f57'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.160185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ab37262675ab43e9ad17696d35567f57'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.161005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ab37262675ab43e9ad17696d35567f57'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.161688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ab37262675ab43e9ad17696d35567f57'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.169545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ab37262675ab43e9ad17696d35567f57'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.177554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 978/1940] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:30.215323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 963/1941] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:29 2022] DELETE /identity/v3/users/ab37262675ab43e9ad17696d35567f57 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:30.432939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:30.434370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.494109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc0e2d-c921-472d-a5b9-a75a4b213acc service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:30.527166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:30.544290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:30.544911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.545192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.545521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.547303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.579919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:30.582197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.627197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:30.628681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f9b630-34d1-4a0e-8bcd-36e7edef41e5 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:30.648662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 979/1942] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:30.670376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb0e08-3c5d-4fdc-b55a-8ea319c34554 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:30.675710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:30.677108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.677476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.677765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] PATH_INFO: `/v3/projects/7a324515436b402097b99949c2497e42` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.679085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a324515436b402097b99949c2497e42)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:30.681049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:30.776962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:30.783917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a324515436b402097b99949c2497e42', 'request_id': 'req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.793861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a324515436b402097b99949c2497e42', 'request_id': 'req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.803100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a324515436b402097b99949c2497e42', 'request_id': 'req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.815212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] The token cache is being invalidate because project 7a324515436b402097b99949c2497e42 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:30.815613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.816343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-379f8b7f-c456-4bc4-bf2a-cdde36a4684b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:30.818252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 964/1943] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:30 2022] DELETE /identity/v3/projects/7a324515436b402097b99949c2497e42 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:30.823728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.823999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.824530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.847115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:30.849056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.943584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 980/1944] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:30.945332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:30.945706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:30.945994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:30.971070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:30.973078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:30.983470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 965/1945] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:30.988222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:30.999898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7316b9ee-6a91-4bfa-8840-b32809a56e64 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.005445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.006067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.006478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.006797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.007489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:31.007803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.009494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.028491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 981/1946] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:31.032213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.056069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd52a7-d5a8-49c3-8bcc-1c2750594a4f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.078194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.078885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.079232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.079538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.080348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.083929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.156110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 966/1947] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/projects => generated 365 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:31.163070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.179845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b2f8bed-b06b-4ba3-8e4e-2265094cc79b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.230262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.232477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:31.314794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.316374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.317947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.318491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.320061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.323040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.357398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 982/1948] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/users => generated 283 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:31.361853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.373172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:31.385413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dae1259-83f1-48d0-a4b3-ace78e576ebd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.387654 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ce1615-64dc-4e31-be3f-13fd7d807111 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.393879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.394888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.394888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.395724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] PATH_INFO: `/v3/users/3d2ae20b39d64633a64c007f19fd3fb3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.396963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2ae20b39d64633a64c007f19fd3fb3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.397069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.397705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.397905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.398182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.399262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.399367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.400524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.419932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 983/1949] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:31.424078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.437525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abaf76c-826f-4eb1-9d57-7a609765ccae admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.445788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.446461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.447727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.448131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.449529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.452341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.469355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 984/1950] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:31.473221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.490942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c41924-f862-44a5-8050-c162eef64397 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.494752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2ae20b39d64633a64c007f19fd3fb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:31.495891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.497764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.498294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.498795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] PATH_INFO: `/v3/projects/05b48539b94f472caa2c29760e11288e/users/b14c77d057934ded9a7fd70b9f56b3f3/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.514850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2ae20b39d64633a64c007f19fd3fb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:31.515664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] RBAC: Authorizing `identity:create_grant(project_id=05b48539b94f472caa2c29760e11288e, user_id=b14c77d057934ded9a7fd70b9f56b3f3, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.515812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2ae20b39d64633a64c007f19fd3fb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:31.516585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2ae20b39d64633a64c007f19fd3fb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:31.518909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1110c64b-9adb-4746-a63b-09e3c72e738d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.527761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2e94b9e9-b211-46fb-b08a-88e142679245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2ae20b39d64633a64c007f19fd3fb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:31.539932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 985/1951] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:31 2022] PUT /identity/v3/projects/05b48539b94f472caa2c29760e11288e/users/b14c77d057934ded9a7fd70b9f56b3f3/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:31.553870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.554986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.555386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.557399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 967/1952] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:31 2022] DELETE /identity/v3/users/3d2ae20b39d64633a64c007f19fd3fb3 => generated 0 bytes in 328 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:31.562448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.562737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.563374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.597723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] MFA Rules not processed for user `b14c77d057934ded9a7fd70b9f56b3f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:31.599260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:31.599371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:31.600505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:31.618481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:31.690709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 968/1953] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:31.701570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.707563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 986/1954] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:31.707933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:31.710132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.711673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:31.727180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709ab572-f961-42f7-a2b8-0fab5c9ede1a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.732364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.732803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.733088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.733368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.734061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.736155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.769065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 987/1955] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/projects => generated 363 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:31.773527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.785471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3df59e1-ce7f-4756-a9f7-5843717010f6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.896801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.897476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.897758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.898031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.920378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde93b47-f0a0-4aa6-a6ac-0f86c3e98ed5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:31.925696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:31.926465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:31.926754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:31.927097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:31.927804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.936050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:31.986740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 988/1956] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:31 2022] POST /identity/v3/users => generated 288 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:31.989703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:31.996013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:31.997689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.011251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 969/1957] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:32.017781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63eb31d1-b624-4774-b69d-9376e622842b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.040187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.040992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.041178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.041426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.042140 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:32.043887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.087529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 989/1958] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:32.088399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.107948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d44540-4b30-4710-bd2c-4b6d957250ad service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.123755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.123968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.124195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.125092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] PATH_INFO: `/v3/projects/1490553751e84170a73a63701a255145/users/72997b19183c4fc39b16deada84e3af6/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.173678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1490553751e84170a73a63701a255145, user_id=72997b19183c4fc39b16deada84e3af6, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:32.178087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.250368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.256466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 970/1959] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:32 2022] PUT /identity/v3/projects/1490553751e84170a73a63701a255145/users/72997b19183c4fc39b16deada84e3af6/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:32.256937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:32.293177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc39ed0-e77f-423d-b7a7-af966328482a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.416696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.417365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.417643 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.417928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.418645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:32.420349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cca9bf0-44d4-49f8-8372-3ca89bc1b344 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.448243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 990/1960] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:32 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:32.450588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.457751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:32.472110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251f67b-f03e-43a3-9127-f2ae83482520 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.488194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.490756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.491256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.493034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] PATH_INFO: `/v3/projects/1490553751e84170a73a63701a255145/users/72997b19183c4fc39b16deada84e3af6/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.499397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1490553751e84170a73a63701a255145, user_id=72997b19183c4fc39b16deada84e3af6, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:32.502540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.525831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 971/1961] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:32 2022] PUT /identity/v3/projects/1490553751e84170a73a63701a255145/users/72997b19183c4fc39b16deada84e3af6/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:32.538724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.539095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.539575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.649492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] MFA Rules not processed for user `72997b19183c4fc39b16deada84e3af6`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:32.654672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:32.686881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.688939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:32.813375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9a450c-72c6-4288-90a5-9f13663a31f1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.816387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 991/1962] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:32 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:32.825990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.827965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:32.835713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.837523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.838063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.838703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] PATH_INFO: `/v3/projects/999cecea09824837a577066ed19abea3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.840546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] RBAC: Authorizing `identity:delete_project(project_id=999cecea09824837a577066ed19abea3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:32.843144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:32.907776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e56509-2b38-4a1c-aed4-f91c7839994a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:32.915192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '999cecea09824837a577066ed19abea3', 'request_id': 'req-798394c8-40ee-4d80-95b9-3df8a39eab98'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:32.921647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '999cecea09824837a577066ed19abea3', 'request_id': 'req-798394c8-40ee-4d80-95b9-3df8a39eab98'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:32.933337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '999cecea09824837a577066ed19abea3', 'request_id': 'req-798394c8-40ee-4d80-95b9-3df8a39eab98'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:32.951700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:32.960323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] The token cache is being invalidate because project 999cecea09824837a577066ed19abea3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:32.960630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:32.961406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:32.963403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 972/1963] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:32 2022] DELETE /identity/v3/projects/999cecea09824837a577066ed19abea3 => generated 0 bytes in 278 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:32.967584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:32.969145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:32.969914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:32.970642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:32.971060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:32.971190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:32.972991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:33.035758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798394c8-40ee-4d80-95b9-3df8a39eab98 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:33.037904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:33.039359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:33.040511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:33.041157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:33.041432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:33.041702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] PATH_INFO: `/v3/users/dc589d6cbc414b318f300f36c351b09e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:33.042833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc589d6cbc414b318f300f36c351b09e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:33.044815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:33.049209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 992/1964] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:32 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:33.066345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:33.095437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc589d6cbc414b318f300f36c351b09e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.108897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc589d6cbc414b318f300f36c351b09e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.109650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc589d6cbc414b318f300f36c351b09e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.110281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc589d6cbc414b318f300f36c351b09e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.117291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc589d6cbc414b318f300f36c351b09e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.129403 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:33.131580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:33.134507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 973/1965] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:32 2022] DELETE /identity/v3/users/dc589d6cbc414b318f300f36c351b09e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:33.144902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e36250-b405-4769-9228-185dbbf320d0 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:33.212480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:33.215691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:33.216280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:33.216831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:33.220607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:33.313458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:33.316596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb914db-41bb-4b7a-a9da-7665d8afadc2 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:33.344978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 993/1966] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:33.716040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:33.718036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:33.849600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9088b56-6ef5-4611-8348-eb4224f8d2dc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:33.857711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:33.858473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:33.858858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:33.859208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] PATH_INFO: `/v3/projects/3408ac4c39464c57a526e22e3047eb02` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:33.860985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=3408ac4c39464c57a526e22e3047eb02)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:33.865290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:33.934403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3408ac4c39464c57a526e22e3047eb02', 'request_id': 'req-ff7638c9-252b-41e7-92ba-44f71e48d0ab'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.948304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3408ac4c39464c57a526e22e3047eb02', 'request_id': 'req-ff7638c9-252b-41e7-92ba-44f71e48d0ab'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.976649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3408ac4c39464c57a526e22e3047eb02', 'request_id': 'req-ff7638c9-252b-41e7-92ba-44f71e48d0ab'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:33.999999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] The token cache is being invalidate because project 3408ac4c39464c57a526e22e3047eb02 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:34.000297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:34.001040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff7638c9-252b-41e7-92ba-44f71e48d0ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:34.008057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 974/1967] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:33 2022] DELETE /identity/v3/projects/3408ac4c39464c57a526e22e3047eb02 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:34.009552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.010051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.010139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.048797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:34.051952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:34.138517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:34.178871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 994/1968] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:34.192866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.193397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.194096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.219034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:34.220443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:34.229461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 975/1969] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:34.233876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.244988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42647c0-8e73-48e2-b8af-7a5bf8463db8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.250849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.251513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.251904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.252129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.252841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:34.253145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.255335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.269738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 995/1970] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:34.273700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.283728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765ed74-2934-4a38-bd85-2a8e855d7e84 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.288431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.289052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.289334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.289609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.290274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.292226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.314630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:34.324522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 976/1971] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/projects => generated 379 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:34.329369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.340225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19a88a7-f45b-40b4-860b-c2c44f035f0d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.401327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.401851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.402179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.402486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.403116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.405083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.430439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 996/1972] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:34.431980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.443515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911e2ecd-a3b5-4aa8-b14a-d7a491316bed admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.448935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.449710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.450453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.450830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.451678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.453403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.470226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 977/1973] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:34.471353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.486266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1346d2-01ae-4f82-a9d9-8d98c6564d8b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.492412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.493040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.493316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.493586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.494288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.495934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e345385-b487-4b34-ac53-d0fdcb19171d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.506144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 997/1974] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:34.510525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.521528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27895e03-64bd-4481-bb0c-efdd8fff6931 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.526816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.527515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.527798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.528108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] PATH_INFO: `/v3/projects/26ad3880ee3848978abe88e5979147d0/users/c791d7d6686f4b6f9d29cbb4243fba0b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.544505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=26ad3880ee3848978abe88e5979147d0, user_id=c791d7d6686f4b6f9d29cbb4243fba0b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.547189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.566047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 978/1975] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:34 2022] PUT /identity/v3/projects/26ad3880ee3848978abe88e5979147d0/users/c791d7d6686f4b6f9d29cbb4243fba0b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:34.580896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.581180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.581452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.609840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] MFA Rules not processed for user `c791d7d6686f4b6f9d29cbb4243fba0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:34.611056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:34.682432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 998/1976] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:34.686719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.760340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d60b9e-674d-43ef-bee0-c0248efec2b8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.765892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:34.766643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:34.767027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:34.767455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:34.768716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:34.771172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:34.826207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 979/1977] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/projects => generated 379 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:34.830331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:34.848899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405175fa-a5b9-45d1-8c1c-1d5802ef91f9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:34.955213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:35.040209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.044992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.045384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.046027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.046838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.049392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.099530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 999/1978] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:34 2022] POST /identity/v3/users => generated 290 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:35.100757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.112716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd2eb15-5b42-4604-bb94-6d412b69dc1c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.131816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.132471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.133233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.133517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.134454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.136031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.151000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.153023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:35.171882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 980/1979] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:35.172867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.178844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd50370-114b-4e78-a2a5-7d1b731718ec admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.193676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52c467d5-9299-41d4-a916-9792bd51d03b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.194490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.195327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.196018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.196443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] PATH_INFO: `/v3/users/03043fdf8fc943fe8334c8aa6052d298` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.198904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=03043fdf8fc943fe8334c8aa6052d298)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.201364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.202125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.202591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.202871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.204281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.205888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.207573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.221846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 981/1980] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:35.222878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.252441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445e4cc5-3f96-408e-919d-51d9a12b9cbf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.258992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.259826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.260347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.261359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] PATH_INFO: `/v3/projects/1dc6fb3bec3a4cd58cec42bbf3735348/users/6e6a2e5f3d0149189d7c880c027335fd/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.285257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dc6fb3bec3a4cd58cec42bbf3735348, user_id=6e6a2e5f3d0149189d7c880c027335fd, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.298587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61868e90-4960-42cd-a1d0-b635da4f74ca admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.323139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 982/1981] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:35 2022] PUT /identity/v3/projects/1dc6fb3bec3a4cd58cec42bbf3735348/users/6e6a2e5f3d0149189d7c880c027335fd/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:35.337247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.337663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.338043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.352027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '03043fdf8fc943fe8334c8aa6052d298'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:35.382144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '03043fdf8fc943fe8334c8aa6052d298'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:35.383631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '03043fdf8fc943fe8334c8aa6052d298'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:35.391179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] MFA Rules not processed for user `6e6a2e5f3d0149189d7c880c027335fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:35.397159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:35.402012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '03043fdf8fc943fe8334c8aa6052d298'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:35.402767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c6f60083-2145-4578-a7e6-d650519cce8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '03043fdf8fc943fe8334c8aa6052d298'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:35.433546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1000/1982] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:35 2022] DELETE /identity/v3/users/03043fdf8fc943fe8334c8aa6052d298 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:35.437948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.438230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.438616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.483769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:35.485009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:35.514403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:35.543226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 983/1983] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:35 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:35.548806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.551922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:35.669705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1001/1984] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:35.677837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.679831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:35.717799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90818e15-6e78-4d86-8f8a-254a8ad6abbc None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.725112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.725789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.726104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.726422 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.727215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.729655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.763291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:35.781232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 984/1985] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:35 2022] POST /identity/v3/projects => generated 377 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:35.784585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.784912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfac987-47f6-4e2f-9b9e-8fe94bd5b2c6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.866474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afc3bd-b4e4-47ff-840e-0e7e46c65596 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.869283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.869982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.870326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.870714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.874058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.875965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.876347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.876739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.877580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.880029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.886771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.888412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.903920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1002/1986] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:35.914913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 985/1987] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:35 2022] POST /identity/v3/users => generated 295 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:35.919006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.932313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ba84f-c698-4005-bdac-ec30379487a7 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.939390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.940130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.940512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.940884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:35.942840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:35.944967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:35.963875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1003/1988] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:35 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:35.965467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:35.977986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815a0600-faac-4161-bc33-b3ffe59ccb02 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:35.987063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:35.988126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:35.988410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:35.988690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] PATH_INFO: `/v3/projects/7a58c3a337e444e68c61b34f61c50e31/users/206ec5d7393049e5a233970ca4c540c0/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.012881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a58c3a337e444e68c61b34f61c50e31, user_id=206ec5d7393049e5a233970ca4c540c0, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.015976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.039456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 986/1989] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:35 2022] PUT /identity/v3/projects/7a58c3a337e444e68c61b34f61c50e31/users/206ec5d7393049e5a233970ca4c540c0/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:36.045436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.057167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22edb077-3cd2-4767-8086-9fc44dcfbbd2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:36.151786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:36.152505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.152894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.153189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.154053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.156314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98814f15-9287-41d1-823b-e8b04bc740da admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.174758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1004/1990] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:36 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:36.176946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.194409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d585908d-bcf0-439f-8acc-d29d1b03e731 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:36.199016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:36.199716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.200250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.200527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] PATH_INFO: `/v3/projects/7a58c3a337e444e68c61b34f61c50e31/users/206ec5d7393049e5a233970ca4c540c0/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.203082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a58c3a337e444e68c61b34f61c50e31, user_id=206ec5d7393049e5a233970ca4c540c0, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.205748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.221527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 987/1991] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:36 2022] PUT /identity/v3/projects/7a58c3a337e444e68c61b34f61c50e31/users/206ec5d7393049e5a233970ca4c540c0/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:36.231460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.231797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.231932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.261218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] MFA Rules not processed for user `206ec5d7393049e5a233970ca4c540c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:36.262438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.351995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1005/1992] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:36 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:36.357231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.359408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.413073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:36.466337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.468207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.484725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216d0f9a-ca83-4bd7-8611-aeeb49a9ceb2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:36.500641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:36.502772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.503148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.503534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.508928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:36.517340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.519109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.537281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 988/1993] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:36 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:36.603398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.606018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.625632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb41156d-b3e4-4ef4-a90f-7f2cf70f068f service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:36.631284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:36.633788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.634324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.635625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc0bbaf-be7a-4bde-9025-75e99307f4cb None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:36.636500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.639314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.641569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:36.642343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:36.642724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:36.643100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] PATH_INFO: `/v3/projects/03f653b370f24cea916a4dac5cf8555b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:36.644548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03f653b370f24cea916a4dac5cf8555b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.649087 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.769126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03f653b370f24cea916a4dac5cf8555b', 'request_id': 'req-e856ce01-85f2-45c1-9b8b-9b02901dde04'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:36.780544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03f653b370f24cea916a4dac5cf8555b', 'request_id': 'req-e856ce01-85f2-45c1-9b8b-9b02901dde04'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:36.789779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03f653b370f24cea916a4dac5cf8555b', 'request_id': 'req-e856ce01-85f2-45c1-9b8b-9b02901dde04'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:36.801924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:36.803402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:36.805409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] The token cache is being invalidate because project 03f653b370f24cea916a4dac5cf8555b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:36.805713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:36.806481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:36.808422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1006/1994] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:36 2022] DELETE /identity/v3/projects/03f653b370f24cea916a4dac5cf8555b => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:36.812565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:36.814411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:36.835990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 989/1995] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 233 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:37.009965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e856ce01-85f2-45c1-9b8b-9b02901dde04 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:37.019680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:37.021395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:37.021873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:37.022356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] PATH_INFO: `/v3/users/8968abe08d3441e2960f2bae06d7682d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:37.023926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8968abe08d3441e2960f2bae06d7682d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:37.026914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:37.141948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:37.148866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8968abe08d3441e2960f2bae06d7682d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.179454 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8968abe08d3441e2960f2bae06d7682d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.179902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8968abe08d3441e2960f2bae06d7682d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.194138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8968abe08d3441e2960f2bae06d7682d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.194880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64c5a115-4eaf-4dbc-aef8-f9eb6bfbeec9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8968abe08d3441e2960f2bae06d7682d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.217641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1007/1996] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:36 2022] DELETE /identity/v3/users/8968abe08d3441e2960f2bae06d7682d => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:37.285991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:37.287659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:37.316181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c335e1ca-8fa8-4895-9596-39ceb3370bc7 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:37.545297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:37.546150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:37.546440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:37.546722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] PATH_INFO: `/v3/users/b14c77d057934ded9a7fd70b9f56b3f3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:37.548999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b14c77d057934ded9a7fd70b9f56b3f3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:37.551008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:37.690791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b14c77d057934ded9a7fd70b9f56b3f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.717191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b14c77d057934ded9a7fd70b9f56b3f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.718515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b14c77d057934ded9a7fd70b9f56b3f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.719481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b14c77d057934ded9a7fd70b9f56b3f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.737497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b14c77d057934ded9a7fd70b9f56b3f3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:37.742889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:37.779643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 990/1997] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:37 2022] DELETE /identity/v3/users/b14c77d057934ded9a7fd70b9f56b3f3 => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:37.784376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:37.784665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:37.784988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:37.853093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:37.854372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:37.975611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1008/1998] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:38.001462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:38.003872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:38.093905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be1d5c-d2f2-4b72-92d6-f973aca8d707 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:38.099750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:38.101245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:38.101675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:38.101964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:38.140589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:38.145709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:38.158380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 991/1999] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:38.186639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:38.188185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:38.203819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e137f29-b515-47dc-936d-485da09cac7a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:38.210823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:38.213771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:38.214162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:38.214545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] PATH_INFO: `/v3/projects/039ba2ed685f4b9ea69861ecd62e087e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:38.216219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] RBAC: Authorizing `identity:delete_project(project_id=039ba2ed685f4b9ea69861ecd62e087e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:38.220647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:38.302468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '039ba2ed685f4b9ea69861ecd62e087e', 'request_id': 'req-5f83b731-f0dc-481a-997a-c0a418244251'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.312746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '039ba2ed685f4b9ea69861ecd62e087e', 'request_id': 'req-5f83b731-f0dc-481a-997a-c0a418244251'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.321304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '039ba2ed685f4b9ea69861ecd62e087e', 'request_id': 'req-5f83b731-f0dc-481a-997a-c0a418244251'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.334541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] The token cache is being invalidate because project 039ba2ed685f4b9ea69861ecd62e087e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:38.334975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.336475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.338981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1009/2000] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:38 2022] DELETE /identity/v3/projects/039ba2ed685f4b9ea69861ecd62e087e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:38.343020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:38.354975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c2ea81-b6d3-4779-bb4c-58bf61066d00 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:38.436773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:38.437533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:38.437940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:38.438281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] PATH_INFO: `/v3/users/1d06405c59f84572b47a7f3ee4ac816e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:38.439586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d06405c59f84572b47a7f3ee4ac816e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:38.441661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:38.495315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d06405c59f84572b47a7f3ee4ac816e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.509960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d06405c59f84572b47a7f3ee4ac816e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.510708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d06405c59f84572b47a7f3ee4ac816e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.515823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d06405c59f84572b47a7f3ee4ac816e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.523628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d06405c59f84572b47a7f3ee4ac816e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.546405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 992/2001] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:38 2022] DELETE /identity/v3/users/1d06405c59f84572b47a7f3ee4ac816e => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:38.561421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:38.562984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:38.577777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83b731-f0dc-481a-997a-c0a418244251 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:38.642839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=11,keystone:SELECT=87,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:38.663190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:38.664625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:38.665327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:38.665666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] PATH_INFO: `/v3/projects/05b48539b94f472caa2c29760e11288e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:38.667193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] RBAC: Authorizing `identity:delete_project(project_id=05b48539b94f472caa2c29760e11288e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:38.669655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:38.749578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05b48539b94f472caa2c29760e11288e', 'request_id': 'req-84330963-382c-455a-8bf7-9d458856698c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.759091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05b48539b94f472caa2c29760e11288e', 'request_id': 'req-84330963-382c-455a-8bf7-9d458856698c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.767801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05b48539b94f472caa2c29760e11288e', 'request_id': 'req-84330963-382c-455a-8bf7-9d458856698c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.782507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] The token cache is being invalidate because project 05b48539b94f472caa2c29760e11288e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:38.782836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.783736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:38.786651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1010/2002] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:38 2022] DELETE /identity/v3/projects/05b48539b94f472caa2c29760e11288e => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:38.790428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:38.792039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:38.829706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae738ee-50d4-4a3f-b1cd-82e0b8972fc8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:38.844120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=2,keystone:DELETE=11,keystone:SELECT=87 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:38.937102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:38.937947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:38.938809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:38.939197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] PATH_INFO: `/v3/users/72997b19183c4fc39b16deada84e3af6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:38.941266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72997b19183c4fc39b16deada84e3af6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:38.944689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.065172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72997b19183c4fc39b16deada84e3af6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.065748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.067282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:39.090989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84330963-382c-455a-8bf7-9d458856698c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.093734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72997b19183c4fc39b16deada84e3af6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.099253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72997b19183c4fc39b16deada84e3af6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.100155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72997b19183c4fc39b16deada84e3af6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.111631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.112296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.112576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.112880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] PATH_INFO: `/v3/projects/55855b449c8e4f3f8de99f39719f9837` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.114670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55855b449c8e4f3f8de99f39719f9837)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.114903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2c6ddf08-9b98-4849-82d5-031069b81d62 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72997b19183c4fc39b16deada84e3af6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.116608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.154798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 993/2003] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:38 2022] DELETE /identity/v3/users/72997b19183c4fc39b16deada84e3af6 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:39.219303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55855b449c8e4f3f8de99f39719f9837', 'request_id': 'req-058825ae-7bef-47d9-8887-7f5f47320245'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.240924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55855b449c8e4f3f8de99f39719f9837', 'request_id': 'req-058825ae-7bef-47d9-8887-7f5f47320245'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.248382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55855b449c8e4f3f8de99f39719f9837', 'request_id': 'req-058825ae-7bef-47d9-8887-7f5f47320245'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.278500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] The token cache is being invalidate because project 55855b449c8e4f3f8de99f39719f9837 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:39.279248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.280699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-058825ae-7bef-47d9-8887-7f5f47320245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:39.283502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1011/2004] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:39 2022] DELETE /identity/v3/projects/55855b449c8e4f3f8de99f39719f9837 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:39.289324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.289623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.289907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.328899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:39.330129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:39.468179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 994/2005] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:39.470091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.470662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.471290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.505817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:39.507452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:39.546083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.565335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1012/2006] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:39.570701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aabbca1-f9e0-4a4f-98ac-6af0d1c01cbd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.588699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.589487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.589778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.590283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.593066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:39.593400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.599795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.617197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.623641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 995/2007] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:39.627596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedd2e02-6296-4e6f-95bf-b2cdfd03cd9a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.632752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.633384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.633828 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.634264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.634943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.637557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.671941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1013/2008] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:39 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:39.676031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.687346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9766eb8c-ba8b-4a90-9f5f-772935f50e0e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.719284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:39.779650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.780299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.780585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.780863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.781593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.783835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.808945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 996/2009] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:39 2022] POST /identity/v3/users => generated 286 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:39.813683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.828304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33494e99-bed0-4ea2-aebc-8fbc92958a5f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.833729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.834561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.835040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.835598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.836378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.840536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.887555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1014/2010] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:39.891837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.904964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996562b-b1e4-4b22-a434-ea6f926c33ab admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.910025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.911056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.911056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.911329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.912200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.913900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66aeaa6d-9567-4dbe-9a6f-5fd6385c7e2e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.927479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 997/2011] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:39.930099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:39.944026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b49a61-577f-468c-bda1-07167581978f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:39.950491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:39.951187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:39.951497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:39.951781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] PATH_INFO: `/v3/projects/b8656d0df1af4475b502efe677f7308e/users/d33866c1c77541a4bfd12d441e7ab175/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:39.975996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8656d0df1af4475b502efe677f7308e, user_id=d33866c1c77541a4bfd12d441e7ab175, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:39.980201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:39.983605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:40.001841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1015/2012] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:39 2022] PUT /identity/v3/projects/b8656d0df1af4475b502efe677f7308e/users/d33866c1c77541a4bfd12d441e7ab175/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:40.007462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.008029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.008392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.028379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:40.029804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:40.098315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 998/2013] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:40.105315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:40.108750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:40.186742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c5ee3f-80e7-42cb-92fc-c8457d680e63 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:40.190727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:40.191382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.192921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.193205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.203237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:40.204864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:40.216006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1016/2014] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:40.700675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:40.702178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:40.714343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020e9d01-bf36-4209-8962-ec31e3c7f315 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:40.719314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:40.720030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.720997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.720997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] PATH_INFO: `/v3/users/c791d7d6686f4b6f9d29cbb4243fba0b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.721997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] RBAC: Authorizing `identity:delete_user(user_id=c791d7d6686f4b6f9d29cbb4243fba0b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:40.724890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:40.777193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:40.780542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:40.788953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c791d7d6686f4b6f9d29cbb4243fba0b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.802903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e7c804-232a-4bf5-947a-2b552f92e008 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:40.805840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c791d7d6686f4b6f9d29cbb4243fba0b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.806610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c791d7d6686f4b6f9d29cbb4243fba0b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.808228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c791d7d6686f4b6f9d29cbb4243fba0b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.810302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:40.811363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.811681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.811962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] PATH_INFO: `/v3/projects/1490553751e84170a73a63701a255145` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.813169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1490553751e84170a73a63701a255145)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:40.815153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:40.819140 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2e43a08-fd4f-4122-9f09-6b9a2c1fe36c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c791d7d6686f4b6f9d29cbb4243fba0b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.841604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 999/2015] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:40 2022] DELETE /identity/v3/users/c791d7d6686f4b6f9d29cbb4243fba0b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:40.846448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ec598-d27b-4bb8-bfe2-87afd162d263 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.846961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ec598-d27b-4bb8-bfe2-87afd162d263 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.848927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9ec598-d27b-4bb8-bfe2-87afd162d263 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.850779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1000/2016] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:23:40 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:23:40.853530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16dc4c-9dcf-40bf-ba91-c73b6200de31 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.853831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16dc4c-9dcf-40bf-ba91-c73b6200de31 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.854279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f16dc4c-9dcf-40bf-ba91-c73b6200de31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.897649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1490553751e84170a73a63701a255145', 'request_id': 'req-a5e5fa62-bfbe-43a1-9021-229210224799'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.905040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0f16dc4c-9dcf-40bf-ba91-c73b6200de31 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:40.906772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0f16dc4c-9dcf-40bf-ba91-c73b6200de31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:40.915692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:40.919166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1490553751e84170a73a63701a255145', 'request_id': 'req-a5e5fa62-bfbe-43a1-9021-229210224799'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.936633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1490553751e84170a73a63701a255145', 'request_id': 'req-a5e5fa62-bfbe-43a1-9021-229210224799'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.967812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] The token cache is being invalidate because project 1490553751e84170a73a63701a255145 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:40.968201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.969043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a5e5fa62-bfbe-43a1-9021-229210224799 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:40.977371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1017/2017] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:40 2022] DELETE /identity/v3/projects/1490553751e84170a73a63701a255145 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:40.977371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371d45-4726-49c6-8340-ccd0a0a129ec None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.977371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371d45-4726-49c6-8340-ccd0a0a129ec None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.977371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d371d45-4726-49c6-8340-ccd0a0a129ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:40.979232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1001/2018] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:40.984873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:40.985405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:40.985765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.011717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:41.013092 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.020779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6d371d45-4726-49c6-8340-ccd0a0a129ec None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:41.023521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6d371d45-4726-49c6-8340-ccd0a0a129ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.033481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1002/2019] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:41.033481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.033481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.118730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1018/2020] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:23:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:41.120684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.120684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.120921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.134935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8aac75-da01-4085-b5e6-11ca3d850871 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.144657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:41.146676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.148357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.151821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.152342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.152839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.157200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1019/2021] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:41.162374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.169626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.171500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.173974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a114dc-2406-483d-a08d-ec2d0c406c9c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.179935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.181071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.181561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.181918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.182701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:41.183076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.185007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18012f62-3f84-439f-b8a9-a23ecccd8559 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.187294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1003/2022] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:41.188304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=2,keystone:SELECT=91,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:41.200110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1020/2023] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:23:41.207127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.224871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75c2300b-d1e5-4a63-a153-2002c0930cd3 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.232042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.233062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.233643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.234108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.235456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.237759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.285571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.286133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.286461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.298306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1004/2024] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:41 2022] POST /identity/v3/projects => generated 367 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:41.304911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.326567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] MFA Rules not processed for user `d33866c1c77541a4bfd12d441e7ab175`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:41.328236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.454579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da3fa19-67fd-4fa4-96ac-8ef26db91fc2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.454995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1021/2025] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:41.456712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.460556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.471179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.471862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.472143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.472419 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.474136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.477753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.506963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1005/2026] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:41 2022] POST /identity/v3/users => generated 284 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:41.508105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.509907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.530095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:41.591924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5569bb95-e795-4260-9744-a58daddb72ee admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.597878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.599400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.599823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.600223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.603075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5d8ae6-26e8-4ed6-b46e-a60e01908cc0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.609811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.610719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.611222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.611398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.611910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.612992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.623277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.625750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.638590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1006/2027] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:41.642919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1022/2028] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:41.643948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.661418 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1ab3e-f67f-4740-9752-c0e94785a55c service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.670189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.671568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.672105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.672562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.673629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.676994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.691456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1007/2029] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:41.695130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.710842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eaddce0-f180-4147-9263-61b9d7b954f4 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.716536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.717352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.717680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.717956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.718780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.721978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7454fa-4894-4ec7-a0b0-5098e88d0dcc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.736347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1023/2030] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:41 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:41.741064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:41.757330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a6b7af-130a-48fc-8d75-9a01cee4154e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:41.765780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:41.766631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.768303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.768303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] PATH_INFO: `/v3/projects/e4812f41f6834f6fb12f66efcb43601b/users/9f517404b8ef4cb4b53b633d00c5678d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.788363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e4812f41f6834f6fb12f66efcb43601b, user_id=9f517404b8ef4cb4b53b633d00c5678d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:41.792948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:41.808178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1008/2031] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:41 2022] PUT /identity/v3/projects/e4812f41f6834f6fb12f66efcb43601b/users/9f517404b8ef4cb4b53b633d00c5678d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:41.822210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:41.822210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:41.824554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:41.863398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] MFA Rules not processed for user `9f517404b8ef4cb4b53b633d00c5678d`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:41.864883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:41.997033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:42.034408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1024/2032] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:41 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:42.039034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.122013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9caae9-ff87-44c6-af14-e1262b909a20 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.136033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.137014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.137370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.137714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.143080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.143080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.189637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1009/2033] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:42 2022] POST /identity/v3/projects => generated 369 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:42.195219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.210644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9ecf14-166d-4709-9175-d47e5d28f773 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.305496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.306481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.306766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.307064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.307819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.309846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.314690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.316438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:42.338259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96541da9-742e-4696-9e6b-e5cfc3b20e3d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.347802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.348562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.348882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.349158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] PATH_INFO: `/v3/projects/26ad3880ee3848978abe88e5979147d0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.354520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] RBAC: Authorizing `identity:delete_project(project_id=26ad3880ee3848978abe88e5979147d0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.356691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.362116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1025/2034] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:42 2022] POST /identity/v3/users => generated 291 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:42.366133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.381956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5609b9-9999-41de-b2d4-23fa5f1e4d5a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.391949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.392666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.393016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.393439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.394236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.396573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.411273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1026/2035] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:42.414560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.428467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26ad3880ee3848978abe88e5979147d0', 'request_id': 'req-6ab004b9-2f7c-4f2e-8477-068f0e35607c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.438046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26ad3880ee3848978abe88e5979147d0', 'request_id': 'req-6ab004b9-2f7c-4f2e-8477-068f0e35607c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.443516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cce76-618d-425e-b8c8-60d2a9e998a0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.450518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26ad3880ee3848978abe88e5979147d0', 'request_id': 'req-6ab004b9-2f7c-4f2e-8477-068f0e35607c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.453373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:42.464308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] The token cache is being invalidate because project 26ad3880ee3848978abe88e5979147d0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:42.464525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.465379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.467751 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1010/2036] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:42 2022] DELETE /identity/v3/projects/26ad3880ee3848978abe88e5979147d0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:42.472172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.474269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:42.544293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.546161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.546851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.547783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] PATH_INFO: `/v3/projects/a289313552ca45aea5a735ef1a6b2a29/users/eb4255968f6548b89c9666f091addac8/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.558433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab004b9-2f7c-4f2e-8477-068f0e35607c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.565065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.565833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.566116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.566388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] PATH_INFO: `/v3/users/6e6a2e5f3d0149189d7c880c027335fd` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.567710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6e6a2e5f3d0149189d7c880c027335fd)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.570280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.601839 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a289313552ca45aea5a735ef1a6b2a29, user_id=eb4255968f6548b89c9666f091addac8, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.605644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.627399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1027/2037] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:42 2022] PUT /identity/v3/projects/a289313552ca45aea5a735ef1a6b2a29/users/eb4255968f6548b89c9666f091addac8/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:42.631993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.634691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:42.644115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6e6a2e5f3d0149189d7c880c027335fd'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.680523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6e6a2e5f3d0149189d7c880c027335fd'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.683487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6e6a2e5f3d0149189d7c880c027335fd'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.685240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6e6a2e5f3d0149189d7c880c027335fd'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.705056 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-268446ca-6b09-490e-91ab-e6b81af0c173 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6e6a2e5f3d0149189d7c880c027335fd'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:42.739721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1011/2038] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:42 2022] DELETE /identity/v3/users/6e6a2e5f3d0149189d7c880c027335fd => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:42.749322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.749916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.749916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.761416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1012/2039] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:23:42 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:23:42.763936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:42.770484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:42.788268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:42.795476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d27458-1662-4492-b934-0a52101fec2d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.908956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14df6980-ef2b-4c04-8d6f-d1c7ddbf6d45 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:42.909155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.912241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.913834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.914132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.914862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.917376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:42.918049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.918329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.918605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] PATH_INFO: `/v3/users/95e33684ef7f4712a9dc517b28d29071` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.919446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccc2fa5-19f3-4c99-beee-c469e2393639 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.919881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] RBAC: Authorizing `identity:delete_user(user_id=95e33684ef7f4712a9dc517b28d29071)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:42.921796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:42.959690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1028/2040] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:42 2022] GET /identity/v3/roles => generated 1420 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:42.964811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:42.965103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:42.965415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:42.965810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:43.031179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] MFA Rules not processed for user `dee88b8fbb6645f6a8df6a26c11941af`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:43.034875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.042166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95e33684ef7f4712a9dc517b28d29071'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.076053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95e33684ef7f4712a9dc517b28d29071'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.080435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95e33684ef7f4712a9dc517b28d29071'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.081414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95e33684ef7f4712a9dc517b28d29071'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.106747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b412b056-83af-49d8-ab65-a250107cad3f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95e33684ef7f4712a9dc517b28d29071'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.161517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1029/2041] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:23:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:43.164644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.196442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.199134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1013/2042] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:42 2022] DELETE /identity/v3/users/95e33684ef7f4712a9dc517b28d29071 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:43.214043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.214352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.214749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.259889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47737067-3ce7-46e4-97c6-0b975eafc1ce None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:43.281034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] MFA Rules not processed for user `d33866c1c77541a4bfd12d441e7ab175`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:43.290425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.386379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:43.386606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.386896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.387154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] PATH_INFO: `/v3/projects/a289313552ca45aea5a735ef1a6b2a29/users/eb4255968f6548b89c9666f091addac8/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.394340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a289313552ca45aea5a735ef1a6b2a29, user_id=eb4255968f6548b89c9666f091addac8, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:43.396331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1014/2043] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:43.398215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.400314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3401f58c-4da9-46ad-acf5-923746e3dda1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:43.403128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.430102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1030/2044] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:43 2022] PUT /identity/v3/projects/a289313552ca45aea5a735ef1a6b2a29/users/eb4255968f6548b89c9666f091addac8/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:43.440507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.440983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.442480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.493227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] MFA Rules not processed for user `eb4255968f6548b89c9666f091addac8`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:43.495142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.521425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:43.525128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec7d50e-aa2d-4b12-9973-8ff413e848ba None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:43.593852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:43.633627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1031/2045] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:43 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:43.634870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.637707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.672091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:43.672777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.673182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.673781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.690363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:43.692100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:43.716669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1015/2046] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:43.717361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.719999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:43.757939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd9f024-8e3c-485c-944b-fb027d8fc53b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:43.768975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:43.769669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.770011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.770438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] PATH_INFO: `/v3/projects/1dc6fb3bec3a4cd58cec42bbf3735348` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.777225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dc6fb3bec3a4cd58cec42bbf3735348)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:43.779177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:43.810889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5741c1fd-8650-4a3c-80ce-0cd58dbc6d7b service placement] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:43.818587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:43.819337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:43.820116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:43.820116 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:43.834649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:43.837724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:43.851568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1016/2047] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:43 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:43.861679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dc6fb3bec3a4cd58cec42bbf3735348', 'request_id': 'req-20c64801-f1bf-4ca6-9575-25bb44f6f272'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.871615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dc6fb3bec3a4cd58cec42bbf3735348', 'request_id': 'req-20c64801-f1bf-4ca6-9575-25bb44f6f272'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.878785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dc6fb3bec3a4cd58cec42bbf3735348', 'request_id': 'req-20c64801-f1bf-4ca6-9575-25bb44f6f272'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.913739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] The token cache is being invalidate because project 1dc6fb3bec3a4cd58cec42bbf3735348 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:43.914305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.914794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:43.916421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1032/2048] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:43 2022] DELETE /identity/v3/projects/1dc6fb3bec3a4cd58cec42bbf3735348 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:43.920469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.931329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec8e75f-17d8-4f07-b50d-a86e8be99999 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:43.979146 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:43.980627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:44.014292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:44.016006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:44.016411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:44.017348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] PATH_INFO: `/v3/users/206ec5d7393049e5a233970ca4c540c0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:44.018853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] RBAC: Authorizing `identity:delete_user(user_id=206ec5d7393049e5a233970ca4c540c0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:44.025453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:44.110235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:44.125936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '206ec5d7393049e5a233970ca4c540c0'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.147115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '206ec5d7393049e5a233970ca4c540c0'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.148105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '206ec5d7393049e5a233970ca4c540c0'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.148920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '206ec5d7393049e5a233970ca4c540c0'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.167402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:44.171358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '206ec5d7393049e5a233970ca4c540c0'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.181880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c64801-f1bf-4ca6-9575-25bb44f6f272 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:44.198727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1017/2049] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:43 2022] DELETE /identity/v3/users/206ec5d7393049e5a233970ca4c540c0 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:44.202128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:44.202884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:44.203173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:44.204451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] PATH_INFO: `/v3/projects/479c4bda3bdc41dcb617b778c52deb20` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:44.206483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=479c4bda3bdc41dcb617b778c52deb20)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:44.213437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:44.329254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479c4bda3bdc41dcb617b778c52deb20', 'request_id': 'req-6c482443-e2e1-4456-9514-f9b8891fdbfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.350247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479c4bda3bdc41dcb617b778c52deb20', 'request_id': 'req-6c482443-e2e1-4456-9514-f9b8891fdbfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.370661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479c4bda3bdc41dcb617b778c52deb20', 'request_id': 'req-6c482443-e2e1-4456-9514-f9b8891fdbfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.387717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] The token cache is being invalidate because project 479c4bda3bdc41dcb617b778c52deb20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:44.388223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.389142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.396163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1033/2050] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:43 2022] DELETE /identity/v3/projects/479c4bda3bdc41dcb617b778c52deb20 => generated 0 bytes in 413 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:44.396331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:44.398328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:44.487273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc838022-03e7-4834-9347-13fe9bb4d244 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:44.600043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:44.600919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:44.601203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:44.601633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] PATH_INFO: `/v3/users/4b47dcf1992142609e08f7aaebd9519e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:44.603016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b47dcf1992142609e08f7aaebd9519e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:44.605159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:44.692987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:44.695299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:44.737729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c482443-e2e1-4456-9514-f9b8891fdbfa admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:44.760209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:44.761546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:44.762306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:44.762770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] PATH_INFO: `/v3/projects/7a58c3a337e444e68c61b34f61c50e31` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:44.765475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a58c3a337e444e68c61b34f61c50e31)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:44.769343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b47dcf1992142609e08f7aaebd9519e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.769756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:44.793492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b47dcf1992142609e08f7aaebd9519e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.794834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b47dcf1992142609e08f7aaebd9519e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.796017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b47dcf1992142609e08f7aaebd9519e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.815695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3209730e-45a1-4999-bd21-dfeda2b06f25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b47dcf1992142609e08f7aaebd9519e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.858277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1018/2051] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:44 2022] DELETE /identity/v3/users/4b47dcf1992142609e08f7aaebd9519e => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:44.891752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a58c3a337e444e68c61b34f61c50e31', 'request_id': 'req-97702baa-d37e-490f-8bae-028086f7c63a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.902827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a58c3a337e444e68c61b34f61c50e31', 'request_id': 'req-97702baa-d37e-490f-8bae-028086f7c63a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.911552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a58c3a337e444e68c61b34f61c50e31', 'request_id': 'req-97702baa-d37e-490f-8bae-028086f7c63a'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.924202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] The token cache is being invalidate because project 7a58c3a337e444e68c61b34f61c50e31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:44.924806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.925202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-97702baa-d37e-490f-8bae-028086f7c63a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:44.927013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1034/2052] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:44 2022] DELETE /identity/v3/projects/7a58c3a337e444e68c61b34f61c50e31 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:44.939863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:44.940355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:44.940626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:44.980799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:44.982495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.029068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=87,keystone:INSERT=2,keystone:DELETE=11 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:45.199445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1019/2053] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:45.203084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.203510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.204284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.212619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.215372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.250058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b85523-1bd4-4782-85a5-6ae1141e216d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.251989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:45.255023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.259205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.260386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.260671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.260999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] PATH_INFO: `/v3/projects/2f925e2803b9448397a37c7b053651b8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.262171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2f925e2803b9448397a37c7b053651b8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.267055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.281222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1035/2054] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:45.285601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.299288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d70c97-ade4-458c-9365-3871fb21bba8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.310287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.311214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.312082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.312521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.313595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:45.313907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.315842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.331297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1036/2055] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:23:45.336461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.346931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2f925e2803b9448397a37c7b053651b8', 'request_id': 'req-c26c0075-a345-4577-bdaf-33fe038046c9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:45.349751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26a4a6-96cb-406f-9bda-6b9c0ec53665 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.358287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2f925e2803b9448397a37c7b053651b8', 'request_id': 'req-c26c0075-a345-4577-bdaf-33fe038046c9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:45.370614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2f925e2803b9448397a37c7b053651b8', 'request_id': 'req-c26c0075-a345-4577-bdaf-33fe038046c9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:45.387133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] The token cache is being invalidate because project 2f925e2803b9448397a37c7b053651b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:45.387859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:45.388647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c26c0075-a345-4577-bdaf-33fe038046c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:45.391471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1020/2056] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:45 2022] DELETE /identity/v3/projects/2f925e2803b9448397a37c7b053651b8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:45.391615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:45.396406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9cf9-a165-41c2-b67d-f173e61b0b0e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.397018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9cf9-a165-41c2-b67d-f173e61b0b0e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.397936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ef9cf9-a165-41c2-b67d-f173e61b0b0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.426720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b8ef9cf9-a165-41c2-b67d-f173e61b0b0e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:45.428605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b8ef9cf9-a165-41c2-b67d-f173e61b0b0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.430325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.431169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.431488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.431988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.432975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.434817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.444736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1021/2057] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:45.451096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a73af-2a4c-49db-848d-5927d991336a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.451497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a73af-2a4c-49db-848d-5927d991336a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.451835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-208a73af-2a4c-49db-848d-5927d991336a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.478816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-208a73af-2a4c-49db-848d-5927d991336a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:45.481829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-208a73af-2a4c-49db-848d-5927d991336a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.496298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1022/2058] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:45.501760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a73af-2a4c-49db-848d-5927d991336a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.503881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1037/2059] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/projects => generated 385 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:45.508105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.512511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.520723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-208a73af-2a4c-49db-848d-5927d991336a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.596820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920a09c6-1ffc-41ae-9a1b-b27f9a42adc2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.597466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.598399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.598682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.599063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.599830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:45.600001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.601587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.607481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.608506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.608860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.609254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.609994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.612335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.615773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1023/2060] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:23:45.620179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.633833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0341f027-7742-448a-83a5-9f7f9d8c16d1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.641611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.642536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.642819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.643126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.644077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.646168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.648960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1038/2061] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/users => generated 293 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:45.652506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.668181 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898de75-30bc-42d3-afe4-18f9442cd779 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.673832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.674631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.674959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.675237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.677622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.679688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:45.680704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.701032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1024/2062] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/projects => generated 385 bytes in 78 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:45.702835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1039/2063] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:45.708000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.708784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.727066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86317026-9cf6-41b8-9372-273ff0f5d8e0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.786863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65477a0e-ea80-48e8-b23a-a5d9ce932594 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.795197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.795852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.796134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.796548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.797212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.799231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.807969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.808776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.809063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.809768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.810926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.812716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.824864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1025/2064] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:45.829433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.829814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1040/2065] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:45.833787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.843305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36d0f4a-0a8c-4452-a2e2-0397c0c82ad9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.844616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea17a8e7-3f23-49a7-a251-93a553976640 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.849337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.850006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.850142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.850381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.850676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.850792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.850966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.851249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] PATH_INFO: `/v3/projects/e19d537627cb4c928e722f1ef9325804/users/806cb5918a544bd4aa8839c1b44fd350/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.851585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.853174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.865181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1026/2066] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:45.869358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:45.878245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e19d537627cb4c928e722f1ef9325804, user_id=806cb5918a544bd4aa8839c1b44fd350, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.881687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98eed044-bb73-4814-910e-8dde1d9de078 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.888945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6822-f214-404c-b6f2-7849826a321c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:45.896902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1041/2067] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:45 2022] PUT /identity/v3/projects/e19d537627cb4c928e722f1ef9325804/users/806cb5918a544bd4aa8839c1b44fd350/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:45.910280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.910608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.913324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.945380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:45.948912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] MFA Rules not processed for user `806cb5918a544bd4aa8839c1b44fd350`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:45.950602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:45.962975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:45.963746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:45.964124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:45.964498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:45.965656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:45.968337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:45.983711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1027/2068] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:45 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:45.984860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.002692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481222-2767-439c-9153-b49f2bd744e7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.009860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.010870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.011328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.011884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] PATH_INFO: `/v3/projects/5f463b5af69940189548c73f4895669d/users/3b35b52041b24d0ea8d6ce4e2b0f8e12/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.033901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f463b5af69940189548c73f4895669d, user_id=3b35b52041b24d0ea8d6ce4e2b0f8e12, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.036811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.040704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1042/2069] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:45 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:46.044354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.044354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.054808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1028/2070] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:45 2022] PUT /identity/v3/projects/5f463b5af69940189548c73f4895669d/users/3b35b52041b24d0ea8d6ce4e2b0f8e12/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:46.055920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.060661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11aee33a-7d74-4553-bd8b-9d37a2212e05 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.130200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:46.143473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.143473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.143473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.143473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] PATH_INFO: `/v3/users/9f517404b8ef4cb4b53b633d00c5678d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.145201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9f517404b8ef4cb4b53b633d00c5678d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.147223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.179313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1beae9-fa0d-46ea-a5d9-e13b48bc21cf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.189943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.191089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.191454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.192334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.193721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.196391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d0746c-be1d-4535-baa1-a992c6cd3b42 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.224715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9f517404b8ef4cb4b53b633d00c5678d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.240729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9f517404b8ef4cb4b53b633d00c5678d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.241572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9f517404b8ef4cb4b53b633d00c5678d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.244914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1029/2071] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/projects => generated 383 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:46.246630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd83e-b0da-4c0e-a493-163cc90e07f1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.246968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd83e-b0da-4c0e-a493-163cc90e07f1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.247372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54dd83e-b0da-4c0e-a493-163cc90e07f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.250926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9f517404b8ef4cb4b53b633d00c5678d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.251965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9f517404b8ef4cb4b53b633d00c5678d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.267730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1043/2072] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:46 2022] DELETE /identity/v3/users/9f517404b8ef4cb4b53b633d00c5678d => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:46.268797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.270303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.279734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f54dd83e-b0da-4c0e-a493-163cc90e07f1 None None] MFA Rules not processed for user `3b35b52041b24d0ea8d6ce4e2b0f8e12`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:46.280966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f54dd83e-b0da-4c0e-a493-163cc90e07f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.307656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:46.347306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8308d0fc-4320-4bbb-a51a-63d8a3f54ac5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.349490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1030/2073] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:46.352293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.352953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.353069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.353234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.353501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.353637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.353969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.354169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.356133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.374581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:46.376125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.382617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1044/2074] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/users => generated 298 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:46.383632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.385268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1031/2075] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:46.385405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.386736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.397329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7875ea4-3356-4b4f-bb30-35745047180d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.402074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.402687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.402961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.403230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.403958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.405525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.416076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1032/2076] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:46.416716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.418338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.453042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3316da37-681c-469b-825b-b292cd094619 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.458165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.458725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.459011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.459293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.468969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.470557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.482810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1045/2077] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:46.483705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.485919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250ef57c-612c-4025-b4e3-6a5fda6b57e8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.490455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.491079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.491436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.491736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.495482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ed5b4bf-3bb8-4512-8b67-4930803d650b service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.501382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.502256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.502754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.503179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.503649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] PATH_INFO: `/v3/projects/57e75ed3db3441349a143d046d22c98a/users/cbd65378d7c341c38269c8d04e673c97/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.503840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.522871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e75ed3db3441349a143d046d22c98a, user_id=cbd65378d7c341c38269c8d04e673c97, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.526444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.541165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1033/2078] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:46.547457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1046/2079] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:46 2022] PUT /identity/v3/projects/57e75ed3db3441349a143d046d22c98a/users/cbd65378d7c341c38269c8d04e673c97/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:46.549025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.549157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.550777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.583289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71beb43-1c0a-4eac-a7a6-5a7386c410d5 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.646662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a636492-4cb6-4b39-a043-01671a41e61f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.652938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.653224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.653469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.653744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.654495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.656150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.669005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1047/2080] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:46 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:46.672837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.684311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:46.686525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03993110-d123-4ac7-8f1b-aac9fffea530 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:46.692429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.693699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.693983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.694595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] PATH_INFO: `/v3/projects/57e75ed3db3441349a143d046d22c98a/users/cbd65378d7c341c38269c8d04e673c97/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.697812 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] RBAC: Authorizing `identity:create_grant(project_id=57e75ed3db3441349a143d046d22c98a, user_id=cbd65378d7c341c38269c8d04e673c97, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.701483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613ccc51-0276-4ea8-a606-164629dda916 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.707846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:46.708742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.708980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.709589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] PATH_INFO: `/v3/users/3b35b52041b24d0ea8d6ce4e2b0f8e12` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.710820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b35b52041b24d0ea8d6ce4e2b0f8e12)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:46.713136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:46.717911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1048/2081] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:46 2022] PUT /identity/v3/projects/57e75ed3db3441349a143d046d22c98a/users/cbd65378d7c341c38269c8d04e673c97/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:46.729023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.729315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.729598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.798103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:46.805763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b35b52041b24d0ea8d6ce4e2b0f8e12'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.810038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] MFA Rules not processed for user `cbd65378d7c341c38269c8d04e673c97`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:46.813117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.833956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b35b52041b24d0ea8d6ce4e2b0f8e12'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.835256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b35b52041b24d0ea8d6ce4e2b0f8e12'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.836617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b35b52041b24d0ea8d6ce4e2b0f8e12'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.849051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5d8c5fad-f7be-4212-9c07-bc8d764ad2f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b35b52041b24d0ea8d6ce4e2b0f8e12'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:46.868348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1034/2082] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:46 2022] DELETE /identity/v3/users/3b35b52041b24d0ea8d6ce4e2b0f8e12 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:46.875853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:46.876423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:46.878243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:46.910524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:46.912180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:46.939151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1049/2083] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:46.940261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:46.949513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.005695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1035/2084] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:47.007334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:47.012554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.054343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b83621-8513-4dd1-88c0-b147cb7867a9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:47.061819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:47.063381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:47.063800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:47.064185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] PATH_INFO: `/v3/projects/e4812f41f6834f6fb12f66efcb43601b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:47.065514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e4812f41f6834f6fb12f66efcb43601b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:47.068478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:47.089146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2b4e383-8c90-4dbd-b51e-be92e2805509 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:47.093402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:47.094051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:47.094331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:47.094603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:47.096592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.130488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:47.133956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e4812f41f6834f6fb12f66efcb43601b', 'request_id': 'req-cab5562a-6897-4e7d-a226-ac25dda29dc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.149175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e4812f41f6834f6fb12f66efcb43601b', 'request_id': 'req-cab5562a-6897-4e7d-a226-ac25dda29dc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.159182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e4812f41f6834f6fb12f66efcb43601b', 'request_id': 'req-cab5562a-6897-4e7d-a226-ac25dda29dc8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.176577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] The token cache is being invalidate because project e4812f41f6834f6fb12f66efcb43601b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:47.177046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.178134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.183827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1050/2085] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:46 2022] DELETE /identity/v3/projects/e4812f41f6834f6fb12f66efcb43601b => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:47.183827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:47.186547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.193076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:47.196267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:47.223815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1036/2086] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:47.226862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:47.233800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:47.254512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73daacea-b4e4-4674-a1ff-8624c07cfa41 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:47.304365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab5562a-6897-4e7d-a226-ac25dda29dc8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:47.314095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:47.314095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:47.314095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:47.314095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:47.318213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.380887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:47.381567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:47.381849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:47.382125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] PATH_INFO: `/v3/users/eb4255968f6548b89c9666f091addac8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:47.384156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb4255968f6548b89c9666f091addac8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:47.386212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:47.449486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:47.451280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:47.465585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1051/2087] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:47.481869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb4255968f6548b89c9666f091addac8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.497952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb4255968f6548b89c9666f091addac8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.499298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb4255968f6548b89c9666f091addac8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.500760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb4255968f6548b89c9666f091addac8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.512513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d4b9afcf-46e7-4597-9cdf-a7c3cbd913a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb4255968f6548b89c9666f091addac8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:47.536983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1037/2088] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:47 2022] DELETE /identity/v3/users/eb4255968f6548b89c9666f091addac8 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:47.943700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:47.945175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:47.963893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5565a5d-8fa2-4efb-88e3-65f760cca14f service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.041227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.041967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.042377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.042719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] PATH_INFO: `/v3/projects/5f463b5af69940189548c73f4895669d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.043990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f463b5af69940189548c73f4895669d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.046044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.096694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f463b5af69940189548c73f4895669d', 'request_id': 'req-3328145d-dab7-4408-8df7-ccc4495046e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.104524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f463b5af69940189548c73f4895669d', 'request_id': 'req-3328145d-dab7-4408-8df7-ccc4495046e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.112290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f463b5af69940189548c73f4895669d', 'request_id': 'req-3328145d-dab7-4408-8df7-ccc4495046e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.127808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] The token cache is being invalidate because project 5f463b5af69940189548c73f4895669d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:48.127808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.128461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3328145d-dab7-4408-8df7-ccc4495046e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.131689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1052/2089] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:47 2022] DELETE /identity/v3/projects/5f463b5af69940189548c73f4895669d => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:48.136341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.136808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.137009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.163504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:48.164703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.206990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:48.247542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1038/2090] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:48.253311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.253620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.253922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.260128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.262022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.279135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a999c76-6eaa-45e6-9135-70772d9c1d8f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.284627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.284969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:48.285344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.285631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.285914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] PATH_INFO: `/v3/projects/a289313552ca45aea5a735ef1a6b2a29` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.287123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a289313552ca45aea5a735ef1a6b2a29)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.289192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.292506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.306491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1053/2091] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:48.307884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.313051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:48.321075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ebd2a8d-b9ff-4566-8b3a-da210d0e9493 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.327966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.329650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.329650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.329650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.330422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:48.330597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.332548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.342737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a289313552ca45aea5a735ef1a6b2a29', 'request_id': 'req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.347471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1054/2092] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:23:48.350066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a289313552ca45aea5a735ef1a6b2a29', 'request_id': 'req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.351034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.359156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a289313552ca45aea5a735ef1a6b2a29', 'request_id': 'req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.361576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7449e-4240-419d-ae97-afb209bb8c85 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.373174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] The token cache is being invalidate because project a289313552ca45aea5a735ef1a6b2a29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:48.373952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.374132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-50f6d6f3-cde1-46ae-91f8-9e86bcb0f9b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:48.376165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1039/2093] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:48 2022] DELETE /identity/v3/projects/a289313552ca45aea5a735ef1a6b2a29 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:48.381738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8d30-1c95-49c0-9255-d101eb09265f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.382021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8d30-1c95-49c0-9255-d101eb09265f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.382296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274e8d30-1c95-49c0-9255-d101eb09265f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.424465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-274e8d30-1c95-49c0-9255-d101eb09265f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:48.425760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-274e8d30-1c95-49c0-9255-d101eb09265f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.442523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.444565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.445741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.446025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.447232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.451029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.507889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1055/2094] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/projects => generated 377 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:48.510392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.512561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.527374 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1040/2095] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:48.537891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.538182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.538469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.579115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:48.580407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.619908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922585d8-a961-42bb-984a-b8d3d50a79b6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.630015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.630674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.630952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.631224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.631978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.634654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.665801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1041/2096] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:48.673757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.680000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1056/2097] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:48.682404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.688284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fb9366-7fe2-4cbc-8809-9d5955407b43 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.694732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8d29de-d361-42f4-9ea0-51c19e75daa9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.698818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.699261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.699585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.699864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.700220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.700821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:48.701093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.701196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.701551 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.701936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.702768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.702893 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.704961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.705093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:48.717102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1042/2098] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:48.720902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1057/2099] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:48.721547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.723487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.733448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99444b55-7cdc-496c-8cc8-437ad18c3601 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.733978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e958587-078f-4667-a066-caba71ddffd2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.739445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.739766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.740412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.740553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.740691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.740973 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.741235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.741644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.741773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.742587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.743746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.744517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.760293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1043/2100] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:48.761378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.772279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6234ff-1244-4111-a6e1-bcb4c5ebae96 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.778716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1058/2101] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/projects => generated 393 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:48.779590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.779952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.780266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.780567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.780864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] PATH_INFO: `/v3/projects/40a54b2d44054e73b7eafc1820c9463b/users/d95807135a1d405ebcd8dc39076ec1fd/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.828125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=40a54b2d44054e73b7eafc1820c9463b, user_id=d95807135a1d405ebcd8dc39076ec1fd, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.830806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91aab93a-28c5-4b90-9d67-30b235b380ec admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.834668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:48.856018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1044/2102] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:48 2022] PUT /identity/v3/projects/40a54b2d44054e73b7eafc1820c9463b/users/d95807135a1d405ebcd8dc39076ec1fd/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:48.857669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.857961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.858239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.885629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef60d56-b9a9-4d29-bc16-dd4e39fe4b5e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.893799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] MFA Rules not processed for user `d95807135a1d405ebcd8dc39076ec1fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:48.895083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:48.959649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.960605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.961053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.961508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.962315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.964650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:48.969196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1045/2103] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:48.973791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.985955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56783413-2b19-4aec-976c-60d04557ed5e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:48.990161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1059/2104] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/users => generated 297 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:48.993492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:48.994191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:48.994870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:48.994870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:48.995607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:48.995741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:48.997721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.022488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4e397-3291-45ac-9c74-4914c5c09387 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.028210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.028955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.029412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.030058 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.030604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.032404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.036270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1046/2105] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:48 2022] POST /identity/v3/projects => generated 377 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:49.041976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.045464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1060/2106] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:48 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:49.049218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.077715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef07c2a-d756-4347-8ee9-3447b1bfb904 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.127283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ba6b2b-51b0-47f3-997d-0452cc160657 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.135322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.135996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.136280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.136555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.137239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.139808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.175189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1047/2107] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/users => generated 289 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:49.179552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.181858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.182491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.183720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.184024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.184753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.190916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.195328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b869b2c6-bd85-4871-a071-ab32f9ddb15b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.203578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1061/2108] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:49.206412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.209281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.209940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.210205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.210482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.211230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.212997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.218827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b12789-8e5c-4496-aa1d-64b5d6e22501 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.224655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1048/2109] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:49.228499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.229199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.229487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.229773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] PATH_INFO: `/v3/projects/06e7d1e042894dfdba5f923edc3abc03/users/dc2768e5db8f4568a8ba05375af2b671/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.232064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.244188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:49.248527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=06e7d1e042894dfdba5f923edc3abc03, user_id=dc2768e5db8f4568a8ba05375af2b671, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.249564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89369b9-c921-400f-9eaf-b64e47a2a94f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.251470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.256241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.257172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.257763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.258259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.259348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.262119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c0d5353-df61-434b-b830-a1383cb93f38 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.271317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1062/2110] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:49 2022] PUT /identity/v3/projects/06e7d1e042894dfdba5f923edc3abc03/users/dc2768e5db8f4568a8ba05375af2b671/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:49.276619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1049/2111] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:49.284965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.285133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.286171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.286631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.326579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] MFA Rules not processed for user `dc2768e5db8f4568a8ba05375af2b671`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:49.328084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:49.346115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:49.396351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6d6e34-6ee1-4100-b7be-1a3cd80bbfc9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.401915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.402684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.402972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.403255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] PATH_INFO: `/v3/projects/0c98b16040df4f86b5abed1fcb082676/users/b311827ad0514ac29f1c4057a586bc57/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.418096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1050/2112] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:49.423635 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.425698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0c98b16040df4f86b5abed1fcb082676, user_id=b311827ad0514ac29f1c4057a586bc57, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.428482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3086774c-025c-4968-ac13-d18c1c302847 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.435706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8acc6-19b0-4a48-8347-3d5874d269da None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.447903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1063/2113] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:49 2022] PUT /identity/v3/projects/0c98b16040df4f86b5abed1fcb082676/users/b311827ad0514ac29f1c4057a586bc57/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:49.453590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.454388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.459786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.460079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.460346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.460464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.460882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.463854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.468233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.500311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] MFA Rules not processed for user `b311827ad0514ac29f1c4057a586bc57`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:49.501594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:49.541372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1051/2114] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/projects => generated 393 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:49.541887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.587582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1064/2115] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:49.592357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.634812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c580b1-f93f-430e-910b-220f3d2bcdca admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.640486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.641187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.641540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.641824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.642503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.645301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.668919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1052/2116] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/users => generated 303 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:49.672821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.673768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd93cd13-7839-4506-b098-19844eef5a8f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.678267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.678880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.679156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.679455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.680160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.682079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.683355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca525959-cc9b-4701-8c19-f19c077988a2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.690571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.691187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.692032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.692313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.693437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.695971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.710997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1053/2117] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:49.715584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.725686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1065/2118] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:49.727132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754e9851-c739-4bf3-84df-68d42995a4a7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.729838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.764361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:49.776506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:49.802631 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.803391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.803801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.804154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] PATH_INFO: `/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b/users/ace5058cadd6453e9823cf88a6aba733/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.805531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da2c7f4-9c73-4867-b3f9-13b2ddcd93a7 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.810243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.810889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.811164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.811449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.812149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.814213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.819058 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ab9c2b3a443dba1f580d3d78bf83b, user_id=ace5058cadd6453e9823cf88a6aba733, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.822304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.842498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1054/2119] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:49 2022] PUT /identity/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b/users/ace5058cadd6453e9823cf88a6aba733/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:49.846891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.848330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1066/2120] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:49 2022] POST /identity/v3/users => generated 295 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:49.852341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.870258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfed76e-efe4-4c1d-a567-243d96e12c58 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.940580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8e5cbc-125d-4ed8-a469-5e75f7792f70 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.945658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.946212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.946612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.946897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.947863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.949245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.954210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.954840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.955118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.955398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.956130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.958302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.960982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1055/2121] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:49.966529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.969491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1067/2122] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:49.974237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:49.979772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168639b8-e2d9-4758-a88d-8bc2a809b53d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.984702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef833ed1-d666-4382-82f4-8e970a2da0d3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:49.985176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.985928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.986303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.986652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] PATH_INFO: `/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b/users/ace5058cadd6453e9823cf88a6aba733/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:49.988789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8c8ab9c2b3a443dba1f580d3d78bf83b, user_id=ace5058cadd6453e9823cf88a6aba733, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:49.990362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:49.991341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:49.991760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:49.991878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480393b8-259d-4c3d-9af9-78112ca9d6eb admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:49.992200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] PATH_INFO: `/v3/projects/5854206f65974701adc0d4a54d9968dc/users/a427021df7ff423da9906aebadf5f3a8/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.006091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1056/2123] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:49 2022] PUT /identity/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b/users/ace5058cadd6453e9823cf88a6aba733/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:50.007824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5854206f65974701adc0d4a54d9968dc, user_id=a427021df7ff423da9906aebadf5f3a8, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.010505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.016927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.017218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.017497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.031082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1068/2124] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:49 2022] PUT /identity/v3/projects/5854206f65974701adc0d4a54d9968dc/users/a427021df7ff423da9906aebadf5f3a8/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:50.035130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:50.048664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] MFA Rules not processed for user `ace5058cadd6453e9823cf88a6aba733`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:50.049807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:50.098749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679adb99-5dd9-4f27-bc7a-d21e63aad455 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:50.103545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:50.104171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.104456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.104734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.105507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.107158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.118305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1069/2125] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:50 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:50.119746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1057/2126] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:50 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:50.122433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:50.132479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:50.134530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:50.137893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5918542-8dc8-480c-81e2-0a8b9d2816c1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:50.143557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:50.144232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.145016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.145373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] PATH_INFO: `/v3/projects/5854206f65974701adc0d4a54d9968dc/users/a427021df7ff423da9906aebadf5f3a8/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.147468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5854206f65974701adc0d4a54d9968dc, user_id=a427021df7ff423da9906aebadf5f3a8, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.150117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97175f8f-dfa7-4e90-938d-846dde30ff84 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.161573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1070/2127] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:50 2022] PUT /identity/v3/projects/5854206f65974701adc0d4a54d9968dc/users/a427021df7ff423da9906aebadf5f3a8/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:50.170601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:50.174954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.175329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.175697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.196895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:50.197494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43710967-a3ac-4b44-a048-e36380caefa5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:50.206032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] MFA Rules not processed for user `a427021df7ff423da9906aebadf5f3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:50.208286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:50.254725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:50.255712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.255997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.256271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.284132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1071/2128] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:50.295369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:50.296770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:50.308377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b62153-219a-403b-a3f4-430e71cc919f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:50.311878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:50.312481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.312764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.313043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.317081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.318524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.328228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1058/2129] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:50.370094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.371595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.376629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:50.381362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1072/2130] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:50.385580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2684c68-85bf-47bc-8ccc-46d3bf9e60dc service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:50.389438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:50.389941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:50.390220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:50.390482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:50.447005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:50.448506 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:50.458227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1059/2131] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:50 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:23:51.995206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:52.003831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce475ba2-8e07-46ef-8cf1-6cab59cf1597 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:52.008136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:52.008768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:52.009052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:52.009327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:52.065563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:52.067660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:52.077110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1073/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:52.634024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:52.642596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8899e99e-c581-49f9-bf48-af14f20a15da service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:52.646252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:52.646913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:52.647224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:52.647538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:52.656679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:52.658073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:52.667879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1060/2133] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:52 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:53.266407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:53.275751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b76dde2-9dbd-4f6e-9740-9603ec1ede04 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:53.279504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:53.280145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:53.280420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:53.280697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:53.282578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:53.329703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:53.358979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:53.360565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7863d04-31d8-4e54-b794-d63224bae481 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:53.372831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1074/2134] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:56.169764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:56.195883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:56.234440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e378c-c96f-4245-8cf7-c2e79291e27f service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:56.239325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:56.240682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.240963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.241238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] PATH_INFO: `/v3/users/dc2768e5db8f4568a8ba05375af2b671` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.242403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc2768e5db8f4568a8ba05375af2b671)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:56.244368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:56.300082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc2768e5db8f4568a8ba05375af2b671'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.314031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc2768e5db8f4568a8ba05375af2b671'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.314739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc2768e5db8f4568a8ba05375af2b671'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.315347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc2768e5db8f4568a8ba05375af2b671'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.322848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc2768e5db8f4568a8ba05375af2b671'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.337007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1061/2135] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:56 2022] DELETE /identity/v3/users/dc2768e5db8f4568a8ba05375af2b671 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:56.341838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.342354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.342458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.363789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:56.365205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:56.427703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1075/2136] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:56.437274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:56.438706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:56.495632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb30645-7238-438c-8ad6-46fc70bfc69a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:56.499191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:56.499847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.500128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.500402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.510006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:56.511447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f556ba2-e54d-4e5c-9052-0968c8ef77ba service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:56.521337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1062/2137] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:56.636774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:56.638341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:56.649993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebe7efd-fa21-4826-bac1-f242013b635d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:56.655130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:56.656002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.656351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.656716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] PATH_INFO: `/v3/users/d95807135a1d405ebcd8dc39076ec1fd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.658207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] RBAC: Authorizing `identity:delete_user(user_id=d95807135a1d405ebcd8dc39076ec1fd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:56.660534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:56.712265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd95807135a1d405ebcd8dc39076ec1fd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.725671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd95807135a1d405ebcd8dc39076ec1fd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.726405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd95807135a1d405ebcd8dc39076ec1fd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.737386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd95807135a1d405ebcd8dc39076ec1fd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.738115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd95807135a1d405ebcd8dc39076ec1fd'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:56.753142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1076/2138] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:56 2022] DELETE /identity/v3/users/d95807135a1d405ebcd8dc39076ec1fd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:56.758738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.759340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.759734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.780687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:56.781902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:56.837198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:56.860750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1063/2139] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:56.874004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:56.876693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:56.928534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:56.948593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032843b8-da43-43d0-a555-069ced22efad admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:56.954291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:56.955009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:56.955292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:56.955627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:56.969633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:56.971098 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:56.983462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1077/2140] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:57.004129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:57.005775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:57.018852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70fd8e5-6dc7-4ed8-ae76-8db7b4dafcb2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:57.024861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:57.025869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.026244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.026619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] PATH_INFO: `/v3/projects/06e7d1e042894dfdba5f923edc3abc03` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.027987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=06e7d1e042894dfdba5f923edc3abc03)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:57.030024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:57.104665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '06e7d1e042894dfdba5f923edc3abc03', 'request_id': 'req-601da8df-7a10-4880-a2f0-8e79ff646042'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.111573 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '06e7d1e042894dfdba5f923edc3abc03', 'request_id': 'req-601da8df-7a10-4880-a2f0-8e79ff646042'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.120885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '06e7d1e042894dfdba5f923edc3abc03', 'request_id': 'req-601da8df-7a10-4880-a2f0-8e79ff646042'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.134191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] The token cache is being invalidate because project 06e7d1e042894dfdba5f923edc3abc03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:57.134476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.135225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.136856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1064/2141] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:57 2022] DELETE /identity/v3/projects/06e7d1e042894dfdba5f923edc3abc03 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:57.144480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:57.155680 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f562512-0738-4de8-980d-4ab044c62fe1 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:57.218358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:57.218647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.218933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.219210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] PATH_INFO: `/v3/users/ace5058cadd6453e9823cf88a6aba733` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.220412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=ace5058cadd6453e9823cf88a6aba733)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:57.222337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:57.280797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ace5058cadd6453e9823cf88a6aba733'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.293784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ace5058cadd6453e9823cf88a6aba733'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.294550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ace5058cadd6453e9823cf88a6aba733'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.303640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ace5058cadd6453e9823cf88a6aba733'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.304392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ace5058cadd6453e9823cf88a6aba733'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.317655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1078/2142] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:57 2022] DELETE /identity/v3/users/ace5058cadd6453e9823cf88a6aba733 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:57.462781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:57.464804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:57.478659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601da8df-7a10-4880-a2f0-8e79ff646042 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:57.543288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:57.546620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.546620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.546620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] PATH_INFO: `/v3/projects/40a54b2d44054e73b7eafc1820c9463b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.546620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] RBAC: Authorizing `identity:delete_project(project_id=40a54b2d44054e73b7eafc1820c9463b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:57.547625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:57.596390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '40a54b2d44054e73b7eafc1820c9463b', 'request_id': 'req-0adc36b6-c5ee-4f5c-b704-7723d139ab11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.603346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '40a54b2d44054e73b7eafc1820c9463b', 'request_id': 'req-0adc36b6-c5ee-4f5c-b704-7723d139ab11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.611286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '40a54b2d44054e73b7eafc1820c9463b', 'request_id': 'req-0adc36b6-c5ee-4f5c-b704-7723d139ab11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.618175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:57.623547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] The token cache is being invalidate because project 40a54b2d44054e73b7eafc1820c9463b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:57.624068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.625139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.627252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1065/2143] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:57 2022] DELETE /identity/v3/projects/40a54b2d44054e73b7eafc1820c9463b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:57.631590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:57.633125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:57.647084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8436aad-d3a0-49fb-ade7-0f20ae7f35ab admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:57.681563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:57.683155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:57.730766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:57.731706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.732011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.732944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] PATH_INFO: `/v3/users/b311827ad0514ac29f1c4057a586bc57` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.734190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b311827ad0514ac29f1c4057a586bc57)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:57.737335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:57.741019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:57.775925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adc36b6-c5ee-4f5c-b704-7723d139ab11 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:57.782003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:57.782405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.782678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.782950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] PATH_INFO: `/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.784145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8c8ab9c2b3a443dba1f580d3d78bf83b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:57.787270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:57.801717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b311827ad0514ac29f1c4057a586bc57'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.816126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b311827ad0514ac29f1c4057a586bc57'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.817075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b311827ad0514ac29f1c4057a586bc57'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.825721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b311827ad0514ac29f1c4057a586bc57'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.826563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e26a98d7-dbc6-4083-b9c3-80819fd06b47 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b311827ad0514ac29f1c4057a586bc57'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.844189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1079/2144] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:57 2022] DELETE /identity/v3/users/b311827ad0514ac29f1c4057a586bc57 => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:57.864190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8c8ab9c2b3a443dba1f580d3d78bf83b', 'request_id': 'req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.874515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8c8ab9c2b3a443dba1f580d3d78bf83b', 'request_id': 'req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.886007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8c8ab9c2b3a443dba1f580d3d78bf83b', 'request_id': 'req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.900578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] The token cache is being invalidate because project 8c8ab9c2b3a443dba1f580d3d78bf83b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:57.900867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.901569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e4a3e385-2c71-4ab7-97fd-5ff17b9b6dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:57.903154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1066/2145] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:57 2022] DELETE /identity/v3/projects/8c8ab9c2b3a443dba1f580d3d78bf83b => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:57.908011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:57.908305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:57.908586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:57.936072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:57.937549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.029544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1080/2146] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:58.032101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.032398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.032682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.058764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:58.060382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.071210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1067/2147] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:58.076878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.088246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc2bdb3-be93-4dc8-9276-c7d3cf5fba94 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.093798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.094920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.095462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.095969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.096799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:58.097388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.099989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.114447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1081/2148] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:58.118911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.137016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65ce6fb-1051-4903-9fe1-111b4c3ca465 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.142599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.143703 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.144145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.144601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.146334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.149322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.187568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1068/2149] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/projects => generated 389 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:58.189960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.204207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36dd8c1-baee-4307-8b48-d4ca5ab6996e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.277288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.278337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.278861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.279225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.280031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.282173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.305639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.308022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.313594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:58.319563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1082/2150] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/users => generated 295 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:58.324810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.330414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b92fc-0fad-4fe2-9df8-4e7d64b61f75 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.340762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749d8a88-fa89-412b-88e8-c6bb00c8eb25 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.346526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.346686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.346831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.346831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.347149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] PATH_INFO: `/v3/projects/0c98b16040df4f86b5abed1fcb082676` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.347357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.347698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.348078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.348403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0c98b16040df4f86b5abed1fcb082676)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.348908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.350979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.358260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.375559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1083/2151] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:58.379273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.400976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95b04f2-2985-4ee9-97c8-07001bdd4a90 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.408206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.408837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.417539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.418481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.419237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.425674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.441254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0c98b16040df4f86b5abed1fcb082676', 'request_id': 'req-3257a0ad-f898-4bd8-9329-321cb31cc8d9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.450414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1084/2152] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:58.450642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0c98b16040df4f86b5abed1fcb082676', 'request_id': 'req-3257a0ad-f898-4bd8-9329-321cb31cc8d9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.453200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:58.459715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.463178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0c98b16040df4f86b5abed1fcb082676', 'request_id': 'req-3257a0ad-f898-4bd8-9329-321cb31cc8d9'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.479659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568c676b-e868-4f7f-a2ac-a7e6d8a64850 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.492689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] The token cache is being invalidate because project 0c98b16040df4f86b5abed1fcb082676 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:58.493349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.494123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.497238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1069/2153] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:58 2022] DELETE /identity/v3/projects/0c98b16040df4f86b5abed1fcb082676 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:58.503732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.505935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.602566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3257a0ad-f898-4bd8-9329-321cb31cc8d9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.608074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.608674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.608963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.609241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] PATH_INFO: `/v3/users/a427021df7ff423da9906aebadf5f3a8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.610685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a427021df7ff423da9906aebadf5f3a8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.613426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.626345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.627071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.627343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.627714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] PATH_INFO: `/v3/projects/fdad334496164bb39325e81fcc7e42b9/users/752b082c48a04057a11882557722bdba/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.653739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdad334496164bb39325e81fcc7e42b9, user_id=752b082c48a04057a11882557722bdba, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.657374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.678877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1085/2154] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:58 2022] PUT /identity/v3/projects/fdad334496164bb39325e81fcc7e42b9/users/752b082c48a04057a11882557722bdba/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:58.679119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a427021df7ff423da9906aebadf5f3a8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.679937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.682850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.697576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a427021df7ff423da9906aebadf5f3a8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.698993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a427021df7ff423da9906aebadf5f3a8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.703247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a427021df7ff423da9906aebadf5f3a8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.713098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9058bcc6-1308-4e3c-9557-0b686380d057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a427021df7ff423da9906aebadf5f3a8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.735220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1070/2155] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:58 2022] DELETE /identity/v3/users/a427021df7ff423da9906aebadf5f3a8 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:58.735220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.735220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.735220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.770265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73dbfc78-8669-4878-8328-5ebc88c7fddc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.772332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] MFA Rules not processed for user `752b082c48a04057a11882557722bdba`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:58.773918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.853347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=2,keystone:DELETE=13,keystone:SELECT=85 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:58.858527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.859219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.859573 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.859902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] PATH_INFO: `/v3/users/806cb5918a544bd4aa8839c1b44fd350` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.861305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=806cb5918a544bd4aa8839c1b44fd350)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.863240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.871947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:58.880325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1071/2156] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:58.884697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.886465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:58.902152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb30e6b-677b-48d0-af23-0abeb8528d2a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:58.908373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:58.909353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.909353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.909593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:58.910244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:58.913206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:58.925693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '806cb5918a544bd4aa8839c1b44fd350'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.948150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1072/2157] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/projects => generated 389 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:58.952330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:58.959704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '806cb5918a544bd4aa8839c1b44fd350'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.960435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '806cb5918a544bd4aa8839c1b44fd350'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.975705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '806cb5918a544bd4aa8839c1b44fd350'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.978328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7f07ae08-96ae-4da1-aef2-80678680bd8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '806cb5918a544bd4aa8839c1b44fd350'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:58.993200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1086/2158] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:23:58 2022] DELETE /identity/v3/users/806cb5918a544bd4aa8839c1b44fd350 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:58.997957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:58.998333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:58.998695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.021498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:59.022824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.030662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff55877-e69a-42b0-bc97-37bb540b4e6c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.089274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1087/2159] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:59.094927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.096344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.102019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.102844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.103807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.104276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.105332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.108819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.110367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e877c4-28ca-4989-a5b9-e7728a69da32 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.115599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.116302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.116528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.116806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] PATH_INFO: `/v3/projects/5854206f65974701adc0d4a54d9968dc` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.117994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5854206f65974701adc0d4a54d9968dc)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.120022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.149633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1073/2160] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:58 2022] POST /identity/v3/users => generated 301 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:23:59.151312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.153315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.181560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5854206f65974701adc0d4a54d9968dc', 'request_id': 'req-73f6d38e-4f33-49de-9b09-d172a94dc764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.191123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5854206f65974701adc0d4a54d9968dc', 'request_id': 'req-73f6d38e-4f33-49de-9b09-d172a94dc764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.203549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5854206f65974701adc0d4a54d9968dc', 'request_id': 'req-73f6d38e-4f33-49de-9b09-d172a94dc764'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.217377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] The token cache is being invalidate because project 5854206f65974701adc0d4a54d9968dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:59.217715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.218489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.220393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1088/2161] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:59 2022] DELETE /identity/v3/projects/5854206f65974701adc0d4a54d9968dc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:59.226827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.226827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.240357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:59.242597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f56f83-71e0-41ae-a2c0-01c743aaf28a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.268797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:59.304681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.305313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.305662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.305885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.307739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.310209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f6d38e-4f33-49de-9b09-d172a94dc764 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.315257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.316002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.316358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.317061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.318300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.320206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456e5600-5296-4810-9f28-2138cfc94dd0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.321240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.322663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.331605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1089/2162] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:59.333028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f147a-84cc-4bae-8b45-028967ccbfa9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.334055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1074/2163] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:23:59.334218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f147a-84cc-4bae-8b45-028967ccbfa9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.334510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367f147a-84cc-4bae-8b45-028967ccbfa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.336270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.347719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c88c70-67fb-4827-adbe-601a849fa2a1 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.356817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-367f147a-84cc-4bae-8b45-028967ccbfa9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:59.356981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.357706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.357991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.358266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] PATH_INFO: `/v3/projects/fff53c292ee14e9e9c27068827f7178e/users/2aeb80b6fd564e55b11588dcc08aae2f/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.358847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-367f147a-84cc-4bae-8b45-028967ccbfa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.368576 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1090/2164] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:23:59.377076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.377352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.377436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.382286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff53c292ee14e9e9c27068827f7178e, user_id=2aeb80b6fd564e55b11588dcc08aae2f, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.385616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.404509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1075/2165] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:59 2022] PUT /identity/v3/projects/fff53c292ee14e9e9c27068827f7178e/users/2aeb80b6fd564e55b11588dcc08aae2f/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:23:59.408651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.413672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:59.415127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.506599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1091/2166] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:59.509657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f212f8-5af3-4be8-a52e-eafe98ca3d0a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.511197 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.514825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.515544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.515897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.516957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.517902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.519817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.525372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307542dd-6b8d-440d-9670-2575e05dcc98 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.533227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.533227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.533227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.533227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.533497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:23:59.533699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1076/2167] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 126 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:23:59.535829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.537434 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.540025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.557694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcbcbf6-29c2-4fd9-b6f1-decf3fb1bda1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.570212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1092/2168] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:23:59.572516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.583275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.586164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.586673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.587129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] PATH_INFO: `/v3/projects/fff53c292ee14e9e9c27068827f7178e/users/2aeb80b6fd564e55b11588dcc08aae2f/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.590845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6cb8c-9a8f-4424-a490-148175cb3394 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.592423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fff53c292ee14e9e9c27068827f7178e, user_id=2aeb80b6fd564e55b11588dcc08aae2f, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.598078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7ecfa8-0275-4134-8a36-e208690bd13a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.607450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.607450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.607450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.607450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.607831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.607831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.623709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1077/2169] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:23:59 2022] PUT /identity/v3/projects/fff53c292ee14e9e9c27068827f7178e/users/2aeb80b6fd564e55b11588dcc08aae2f/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:59.636974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.637772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.638844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.643885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1093/2170] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:23:59 2022] POST /identity/v3/projects => generated 399 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:59.649055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.670647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] MFA Rules not processed for user `2aeb80b6fd564e55b11588dcc08aae2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:23:59.671902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.730262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231ff1b6-8ee9-40b3-92e9-9a6767d56b15 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.736853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.737538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.737896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.738245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.739011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.741590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.759920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:59.786963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1094/2171] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:23:59 2022] POST /identity/v3/users => generated 300 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:23:59.788235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.789533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1078/2172] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:23:59 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:23:59.790464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.794025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.810614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96762a25-4f79-4a88-b323-e6555086fa33 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.813991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1447b178-5336-4d03-ba10-af62466f1688 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.815775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.816497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.816859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.817208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] PATH_INFO: `/v3/projects/e19d537627cb4c928e722f1ef9325804` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.819066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e19d537627cb4c928e722f1ef9325804)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.820462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.821082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.821198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.821362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.821627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.822339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:23:59.824015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:23:59.826588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:23:59.835131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1079/2173] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:23:59.840192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.844920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.889970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e19d537627cb4c928e722f1ef9325804', 'request_id': 'req-64ac9919-5cff-454d-b88b-9bfcc99cf816'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.908797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e19d537627cb4c928e722f1ef9325804', 'request_id': 'req-64ac9919-5cff-454d-b88b-9bfcc99cf816'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.916780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60403861-5e39-43ce-b3cf-0bb4ff6a2573 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:23:59.918764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e19d537627cb4c928e722f1ef9325804', 'request_id': 'req-64ac9919-5cff-454d-b88b-9bfcc99cf816'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.934092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] The token cache is being invalidate because project e19d537627cb4c928e722f1ef9325804 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:23:59.934387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.935286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:23:59.937118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1095/2174] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:23:59 2022] DELETE /identity/v3/projects/e19d537627cb4c928e722f1ef9325804 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:23:59.941355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:23:59.942901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:23:59.978474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:23:59.979109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:23:59.979394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:23:59.979699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:23:59.982553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.040955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ac9919-5cff-454d-b88b-9bfcc99cf816 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.046933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.047597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.049180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.049180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.049180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.051494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.064694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.066357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.068949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1096/2175] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:00.069806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.071658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.083350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1080/2176] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:23:59 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:00.084209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.094771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41574c4-9ad0-4238-97da-7dd0704ed85b service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.113567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.115163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.115663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.115957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] PATH_INFO: `/v3/projects/b8ca34831b91428fbc3af1303c9be786/users/016ac28e6ce34a5eb78c62b3a31fe4fc/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.116623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c820f32d-3cbd-4852-89fb-472ea8c973a3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.122494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.123241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.123622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.123964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] PATH_INFO: `/v3/users/cbd65378d7c341c38269c8d04e673c97` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.125380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cbd65378d7c341c38269c8d04e673c97)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.131908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.136798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b8ca34831b91428fbc3af1303c9be786, user_id=016ac28e6ce34a5eb78c62b3a31fe4fc, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.143539 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-322bbb81-0110-41d1-88bc-a46ebbac25aa admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.171289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1081/2177] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:00 2022] PUT /identity/v3/projects/b8ca34831b91428fbc3af1303c9be786/users/016ac28e6ce34a5eb78c62b3a31fe4fc/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:00.181733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.182010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.182442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.207636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cbd65378d7c341c38269c8d04e673c97'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.211839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] MFA Rules not processed for user `016ac28e6ce34a5eb78c62b3a31fe4fc`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:00.213101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.224692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cbd65378d7c341c38269c8d04e673c97'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.225620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cbd65378d7c341c38269c8d04e673c97'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.229503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:INSERT=3,keystone:SELECT=81,keystone:DELETE=16 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:00.236633 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cbd65378d7c341c38269c8d04e673c97'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.238484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cbd65378d7c341c38269c8d04e673c97'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.252122 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:00.254616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1097/2178] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:00 2022] DELETE /identity/v3/users/cbd65378d7c341c38269c8d04e673c97 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:00.255522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.267470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.331142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1082/2179] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:00.352583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.354565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.360125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3a914b-4ceb-4b10-b245-bfb86ff64396 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.370063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.371365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.371679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.371964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] PATH_INFO: `/v3/users/752b082c48a04057a11882557722bdba` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.373353 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=752b082c48a04057a11882557722bdba)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.375688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.429917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c949798d-bc76-4ee1-9de6-3ce85283c48c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.443968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '752b082c48a04057a11882557722bdba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.470430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '752b082c48a04057a11882557722bdba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.471768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '752b082c48a04057a11882557722bdba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.485968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '752b082c48a04057a11882557722bdba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.487177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ee4711c5-6aac-4f89-82fb-1a5feb11c1c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '752b082c48a04057a11882557722bdba'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.508326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1098/2180] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:00 2022] DELETE /identity/v3/users/752b082c48a04057a11882557722bdba => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:00.510200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.510686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.510972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.523207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.524988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.525471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.525962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.528037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.549497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:00.550750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.640850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.643473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.658287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1083/2181] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:00 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:24:00.673412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.679297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.714371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922a4444-318e-4c02-83ab-800b54757851 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.720358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.721164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.721652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.722031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] PATH_INFO: `/v3/projects/57e75ed3db3441349a143d046d22c98a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.723552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1099/2182] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:00.727382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=57e75ed3db3441349a143d046d22c98a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:00.729861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:00.738645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:00.745764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.761605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:00.783249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:00.795559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57e75ed3db3441349a143d046d22c98a', 'request_id': 'req-a44c158b-1404-42a4-be9b-feb455cae9ed'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.804442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57e75ed3db3441349a143d046d22c98a', 'request_id': 'req-a44c158b-1404-42a4-be9b-feb455cae9ed'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.826980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57e75ed3db3441349a143d046d22c98a', 'request_id': 'req-a44c158b-1404-42a4-be9b-feb455cae9ed'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.843057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] The token cache is being invalidate because project 57e75ed3db3441349a143d046d22c98a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:00.843525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.844329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a44c158b-1404-42a4-be9b-feb455cae9ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:00.851651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1084/2183] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:00 2022] DELETE /identity/v3/projects/57e75ed3db3441349a143d046d22c98a => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:00.853439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccdb99-d0e6-4857-af81-d283bd2f6929 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.853902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccdb99-d0e6-4857-af81-d283bd2f6929 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.854305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ccdb99-d0e6-4857-af81-d283bd2f6929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.864204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b6f797-c95b-4a59-b662-21c5e5af1f1d None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:00.880819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a6ccdb99-d0e6-4857-af81-d283bd2f6929 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:00.882455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a6ccdb99-d0e6-4857-af81-d283bd2f6929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.930797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:00.931629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.932024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.932385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:00.934437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:00.968006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1085/2184] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:00.979253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:00.979594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:00.979878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.008931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:01.010679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:01.022450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1086/2185] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:01.027925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.041053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1999182a-b788-4b7d-ab2b-47e4fa48d19f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.043687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.045792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.047641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.048595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.048962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.049336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.050184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:01.050606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.053006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.062419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1100/2186] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:24:01.072097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1087/2187] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:01.077442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.093447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3665fcf5-fdda-4ee0-a815-dc559a5739e0 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.100434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.101255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.102310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.102691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.103614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.106308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.147599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1101/2188] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:01 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:01.152577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.164601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9b49c-8f82-4130-b60f-54652c1a1a40 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.241235 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.241972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.242349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.242690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.243799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.246349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.278432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1088/2189] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:01 2022] POST /identity/v3/users => generated 292 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:01.284405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.300194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89c99cf-4a19-4657-b769-51378926aced admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.307013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.307928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.308580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.308580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.309714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.311369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.328293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1102/2190] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:01.333712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.351283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6237af7b-3616-45df-b04c-265b5af1d100 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.360290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.361439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.362108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.362609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.363903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.367389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a96f095-26a4-4d63-beb9-19e0eb263010 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.388327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1089/2191] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:01 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:01.393645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.409933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d97e47-b863-48e9-9f78-58ad39c3ae1f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.417529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.418274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.418552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.419159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] PATH_INFO: `/v3/projects/8df162614de9479e8327756f9324a19e/users/57aaf5bf00da496ab2ceb38609f4dc93/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.438328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8df162614de9479e8327756f9324a19e, user_id=57aaf5bf00da496ab2ceb38609f4dc93, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.441315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.455307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1103/2192] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:01 2022] PUT /identity/v3/projects/8df162614de9479e8327756f9324a19e/users/57aaf5bf00da496ab2ceb38609f4dc93/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:01.466759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.467175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.467559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.498947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] MFA Rules not processed for user `57aaf5bf00da496ab2ceb38609f4dc93`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:01.500454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:01.509965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:01.568195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1090/2193] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:01.580819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.582766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:01.595182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.596874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:01.605727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:01.650300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6e71a3-efad-483f-87a5-2095b4ac3f2d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.654022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.654672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.654953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.655224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.664006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.665499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.666271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4eda90f-7836-4f7a-ae04-aaea46260023 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.670999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.671673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.671960 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.672240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] PATH_INFO: `/v3/projects/fdad334496164bb39325e81fcc7e42b9` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.673406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdad334496164bb39325e81fcc7e42b9)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.674710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1104/2194] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:01.676206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.724685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdad334496164bb39325e81fcc7e42b9', 'request_id': 'req-35f33d7a-25fb-4f92-8029-6b3a23929017'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.731002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdad334496164bb39325e81fcc7e42b9', 'request_id': 'req-35f33d7a-25fb-4f92-8029-6b3a23929017'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.738961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdad334496164bb39325e81fcc7e42b9', 'request_id': 'req-35f33d7a-25fb-4f92-8029-6b3a23929017'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.750845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] The token cache is being invalidate because project fdad334496164bb39325e81fcc7e42b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:01.751134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.752231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.753348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1091/2195] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:01 2022] DELETE /identity/v3/projects/fdad334496164bb39325e81fcc7e42b9 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:01.757506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:01.768336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e6e87e-c29b-4f2f-9683-1d9a4391608e service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:01.829120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:01.829632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:01.830391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:01.830391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] PATH_INFO: `/v3/users/2aeb80b6fd564e55b11588dcc08aae2f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:01.831856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2aeb80b6fd564e55b11588dcc08aae2f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:01.835025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:01.895109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2aeb80b6fd564e55b11588dcc08aae2f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.912601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2aeb80b6fd564e55b11588dcc08aae2f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.913849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2aeb80b6fd564e55b11588dcc08aae2f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.922019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2aeb80b6fd564e55b11588dcc08aae2f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.922739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98c50398-0c57-4136-8749-fd11e2c80cc9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2aeb80b6fd564e55b11588dcc08aae2f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:01.941296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1105/2196] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:01 2022] DELETE /identity/v3/users/2aeb80b6fd564e55b11588dcc08aae2f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:02.476345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:02.478606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:02.493665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f33d7a-25fb-4f92-8029-6b3a23929017 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:02.586022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:02.610580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:02.611337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:02.611717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:02.612066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] PATH_INFO: `/v3/projects/fff53c292ee14e9e9c27068827f7178e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:02.617111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fff53c292ee14e9e9c27068827f7178e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:02.619188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:02.685704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fff53c292ee14e9e9c27068827f7178e', 'request_id': 'req-779a7cfc-1209-41d7-af6b-2c867e4d3044'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:02.693188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fff53c292ee14e9e9c27068827f7178e', 'request_id': 'req-779a7cfc-1209-41d7-af6b-2c867e4d3044'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:02.706846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fff53c292ee14e9e9c27068827f7178e', 'request_id': 'req-779a7cfc-1209-41d7-af6b-2c867e4d3044'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:02.720110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] The token cache is being invalidate because project fff53c292ee14e9e9c27068827f7178e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:02.720397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:02.721067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-779a7cfc-1209-41d7-af6b-2c867e4d3044 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:02.722608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1092/2197] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:02 2022] DELETE /identity/v3/projects/fff53c292ee14e9e9c27068827f7178e => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:02.728536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:02.728843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:02.729122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:02.750443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:02.752029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:02.787936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:02.838342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1106/2198] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:02.846860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:02.847144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:02.847433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:02.868512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:02.869633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:02.880806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1093/2199] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:02.885277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:02.896182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f491df5-f767-49fc-aef4-9290ee4b1106 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:02.901767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:02.902426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:02.902706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:02.902980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:02.903729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:02.904047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:02.905658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:02.917705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1107/2200] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:02.922208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:02.931692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573b0e3e-66dd-4bb8-866a-9a48fed7bcf8 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:02.936800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:02.937370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:02.937647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:02.937922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:02.938599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:02.940580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:02.972150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1094/2201] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:02 2022] POST /identity/v3/projects => generated 381 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:02.976567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:02.986925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40976dbf-5391-4bf3-b8f7-6e0520978d19 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.057757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.058534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.059321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.059637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.060320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.063220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.107335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1108/2202] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:02 2022] POST /identity/v3/users => generated 291 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:03.112605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.129554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b3b403-e686-4b94-8a57-39a3ae744ff7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.136678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.137315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.137595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.137870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.138583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.140229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.159712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1095/2203] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:03.164231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.183113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d4bb8d-8bf2-46af-8a42-5af25ec19069 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.195600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.196905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.197433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.198224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.200357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.202989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c6b0d1-86f7-4867-854e-98351068c68a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.224721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1109/2204] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:03.228521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.244396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a797175-d39e-4845-8950-9f310bcaea55 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.255393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.256109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.256397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.256675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] PATH_INFO: `/v3/projects/cdf77115dec04230a3a2245089221e33/users/ede56e74e9f542138a36ff7bde17af76/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.292445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cdf77115dec04230a3a2245089221e33, user_id=ede56e74e9f542138a36ff7bde17af76, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.295714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.315378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1096/2205] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:03 2022] PUT /identity/v3/projects/cdf77115dec04230a3a2245089221e33/users/ede56e74e9f542138a36ff7bde17af76/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:03.326529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.327099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.327615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.376221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] MFA Rules not processed for user `ede56e74e9f542138a36ff7bde17af76`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:03.377756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:03.461090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1110/2206] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:03 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:03.465598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.540167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9bf915-f30f-4e6d-ad82-fee69ca35907 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.545639 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.546267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.546549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.546821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.547684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.549949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.566788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:03.589793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1097/2207] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:03 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:03.590928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.602709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae5f22c-aeba-46a6-8565-40cdda2dbc96 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.639078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:03.675120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.676641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.677020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.677330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.679037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.681226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.707398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1111/2208] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:03 2022] POST /identity/v3/users => generated 296 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:03.712022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.726528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519945b5-8788-421c-92ba-2a57b02a5ff8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.732261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.733726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.734368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.734821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.735909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.737805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.756474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1098/2209] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:03.761319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.774991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ee8ffd3-c7d3-4901-835d-ca711675eea3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.780500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.781392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.781851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.782266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] PATH_INFO: `/v3/projects/304621aa750340fcb4dfa0ac0337032e/users/1c5c07f259b6458ca8c5233ab51d23bf/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.800438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=304621aa750340fcb4dfa0ac0337032e, user_id=1c5c07f259b6458ca8c5233ab51d23bf, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.803894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.821464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1112/2210] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:03 2022] PUT /identity/v3/projects/304621aa750340fcb4dfa0ac0337032e/users/1c5c07f259b6458ca8c5233ab51d23bf/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:03.825899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.836457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e85ae6-75c0-4846-bd2d-0ce682f1f30e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.898406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.898955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.899239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.899576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.900269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.901860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9902149-b5f9-4a70-b87f-122f707cac3b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.912627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1099/2211] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:03 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:03.916892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:03.927813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d86c00-33cd-408f-9d51-243fa0864e93 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:03.932890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:03.933544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:03.933821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:03.934212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] PATH_INFO: `/v3/projects/304621aa750340fcb4dfa0ac0337032e/users/1c5c07f259b6458ca8c5233ab51d23bf/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:03.936135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=304621aa750340fcb4dfa0ac0337032e, user_id=1c5c07f259b6458ca8c5233ab51d23bf, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:03.938751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:03.949183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1113/2212] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:03 2022] PUT /identity/v3/projects/304621aa750340fcb4dfa0ac0337032e/users/1c5c07f259b6458ca8c5233ab51d23bf/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:04.018924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:04.019127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:04.019449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:04.049387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] MFA Rules not processed for user `1c5c07f259b6458ca8c5233ab51d23bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:04.050714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:04.131385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1100/2213] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:04.142445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:04.144144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:04.205588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-431c10d5-9019-4109-9e4b-91cfc4284bbe admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:04.209932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:04.210199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:04.210546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:04.210886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:04.221442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:04.223107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86789dfc-778e-4a04-9010-13e16db1079b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:04.233303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1114/2214] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:04 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:04.668384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:04.759507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1da82c-528c-4bb0-a97a-504ea44e6ba7 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:04.776132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:04.776888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:04.777172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:04.777447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] PATH_INFO: `/v3/users/ede56e74e9f542138a36ff7bde17af76` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:04.778826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=ede56e74e9f542138a36ff7bde17af76)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:04.782484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:04.788661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:04.845448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ede56e74e9f542138a36ff7bde17af76'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:04.858104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ede56e74e9f542138a36ff7bde17af76'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:04.858806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ede56e74e9f542138a36ff7bde17af76'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:04.859436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ede56e74e9f542138a36ff7bde17af76'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:04.866730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ede56e74e9f542138a36ff7bde17af76'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:04.887051 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1101/2215] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:04 2022] DELETE /identity/v3/users/ede56e74e9f542138a36ff7bde17af76 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:04.888895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184998-d168-4c05-8176-5d52079ff213 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:04.889364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184998-d168-4c05-8176-5d52079ff213 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:04.889478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04184998-d168-4c05-8176-5d52079ff213 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:04.909322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-04184998-d168-4c05-8176-5d52079ff213 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:04.910553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-04184998-d168-4c05-8176-5d52079ff213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:04.972478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1115/2216] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:04.982619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:04.984131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:05.044967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f7e27a-4f52-46c4-b185-9571b5786f5c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:05.048716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:05.049332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:05.049611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:05.049883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:05.059605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:05.061049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:05.074289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1102/2217] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:05.524020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184998-d168-4c05-8176-5d52079ff213 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:05.529762 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-04184998-d168-4c05-8176-5d52079ff213 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:05.563560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04184998-d168-4c05-8176-5d52079ff213 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:05.570596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:05.571332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:05.571628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:05.571969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] PATH_INFO: `/v3/projects/cdf77115dec04230a3a2245089221e33` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:05.573163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cdf77115dec04230a3a2245089221e33)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:05.577760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:05.617776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:05.648555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cdf77115dec04230a3a2245089221e33', 'request_id': 'req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.662036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cdf77115dec04230a3a2245089221e33', 'request_id': 'req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.672295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cdf77115dec04230a3a2245089221e33', 'request_id': 'req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.687143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] The token cache is being invalidate because project cdf77115dec04230a3a2245089221e33 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:05.687834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.689287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.691596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1116/2218] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:05 2022] DELETE /identity/v3/projects/cdf77115dec04230a3a2245089221e33 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:05.696232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:05.710487 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ac310c-192e-4a2c-bed9-f1488dd3ad44 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:05.795108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:05.796002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:05.796387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:05.796745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] PATH_INFO: `/v3/users/1c5c07f259b6458ca8c5233ab51d23bf` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:05.810177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1c5c07f259b6458ca8c5233ab51d23bf)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:05.820800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:05.910734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:05.955064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1c5c07f259b6458ca8c5233ab51d23bf'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.981262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1c5c07f259b6458ca8c5233ab51d23bf'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.981906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1c5c07f259b6458ca8c5233ab51d23bf'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:05.982525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1c5c07f259b6458ca8c5233ab51d23bf'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.003291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1c5c07f259b6458ca8c5233ab51d23bf'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.033287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1103/2219] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:05 2022] DELETE /identity/v3/users/1c5c07f259b6458ca8c5233ab51d23bf => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:06.374523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:06.376064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:06.377316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:06.380542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:06.405949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19588f88-89bc-4ecf-bc23-ab72a7ff90e5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:06.454922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6792b775-e0dd-4b24-9dda-1396a33212d9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:06.459957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:06.460559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:06.460813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:06.461159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] PATH_INFO: `/v3/projects/304621aa750340fcb4dfa0ac0337032e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:06.462408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] RBAC: Authorizing `identity:delete_project(project_id=304621aa750340fcb4dfa0ac0337032e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:06.464505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:06.466236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:06.466889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:06.467155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:06.467442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] PATH_INFO: `/v3/users/db0c001088aa4c86b4562c369deb6c46` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:06.468619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db0c001088aa4c86b4562c369deb6c46)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:06.470536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:06.516853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '304621aa750340fcb4dfa0ac0337032e', 'request_id': 'req-774a6e41-1bad-42dc-9255-8a2c8c5db98d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.520606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db0c001088aa4c86b4562c369deb6c46'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.523280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '304621aa750340fcb4dfa0ac0337032e', 'request_id': 'req-774a6e41-1bad-42dc-9255-8a2c8c5db98d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.533750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '304621aa750340fcb4dfa0ac0337032e', 'request_id': 'req-774a6e41-1bad-42dc-9255-8a2c8c5db98d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.534568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db0c001088aa4c86b4562c369deb6c46'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.535291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db0c001088aa4c86b4562c369deb6c46'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.543880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db0c001088aa4c86b4562c369deb6c46'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.544693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-11820462-a8e1-4d1d-b1c4-fd3bb66d1524 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db0c001088aa4c86b4562c369deb6c46'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.547667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] The token cache is being invalidate because project 304621aa750340fcb4dfa0ac0337032e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:06.547961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.548585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.550335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1104/2220] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:06 2022] DELETE /identity/v3/projects/304621aa750340fcb4dfa0ac0337032e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:06.559020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1117/2221] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:06 2022] DELETE /identity/v3/users/db0c001088aa4c86b4562c369deb6c46 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:06.754723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:06.756845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:06.802495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:06.834215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774a6e41-1bad-42dc-9255-8a2c8c5db98d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:06.898964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:06.900132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:06.900857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:06.901151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] PATH_INFO: `/v3/projects/f4ef4d34a8114051a67c51e01cb2b8eb` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:06.902338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4ef4d34a8114051a67c51e01cb2b8eb)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:06.904286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:06.955081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4ef4d34a8114051a67c51e01cb2b8eb', 'request_id': 'req-9b0048de-0731-41a1-9920-c5c03fc6a66d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.961754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4ef4d34a8114051a67c51e01cb2b8eb', 'request_id': 'req-9b0048de-0731-41a1-9920-c5c03fc6a66d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.969079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4ef4d34a8114051a67c51e01cb2b8eb', 'request_id': 'req-9b0048de-0731-41a1-9920-c5c03fc6a66d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.979018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] The token cache is being invalidate because project f4ef4d34a8114051a67c51e01cb2b8eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:06.979297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.979938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9b0048de-0731-41a1-9920-c5c03fc6a66d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:06.981454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1105/2222] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:06 2022] DELETE /identity/v3/projects/f4ef4d34a8114051a67c51e01cb2b8eb => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:06.987213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:06.987529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:06.987807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.007129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:07.008341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:07.033014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=17,keystone:SELECT=81,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:07.070237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1118/2223] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:07.077897 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.078185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.078576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.099637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:07.100838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:07.108743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1106/2224] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:07.113302 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.124213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8128aef4-f85d-4029-b161-b1e70a09a2e7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.129089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.130612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.130906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.131182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.131925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:07.132234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.133840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.152926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1119/2225] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:24:07.156919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.168892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abb1c63-e8d4-414f-90b9-13e45a06821f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.173724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.174282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.174561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.177890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.177890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.178097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.208966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1107/2226] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:07 2022] POST /identity/v3/projects => generated 383 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:07.210376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.220939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad64a80-3183-4aa1-bcb4-c9614160971e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.278354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.279051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.279366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.280020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.280702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.282646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.305404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1120/2227] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:07 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:07.309423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.319771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbebd636-51a4-4b04-bca2-3edba5ae9477 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.324398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.325264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.325545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.325821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.326525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.328150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.338906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1108/2228] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:07.342776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.353534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a75cb1-f9aa-4275-a80b-d39342be2e63 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.358359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.358969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.359253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.359570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.360298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.361884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45ef7cdf-1d22-4e0b-a795-2d542b6a60f6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.373225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1121/2229] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:07.377151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.387398 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc430f5a-0345-4b2c-8fd4-3629291cb94c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.392061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.392711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.393010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.393271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] PATH_INFO: `/v3/projects/976675956dca4c50917939c4d660e0b1/users/db1a21e1056443ae9317c997c25d69c6/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.407985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=976675956dca4c50917939c4d660e0b1, user_id=db1a21e1056443ae9317c997c25d69c6, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.410619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.423898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1109/2230] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:07 2022] PUT /identity/v3/projects/976675956dca4c50917939c4d660e0b1/users/db1a21e1056443ae9317c997c25d69c6/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:07.434346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8590f3-d64e-4190-b2f7-f2f297b18c48 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.434854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8590f3-d64e-4190-b2f7-f2f297b18c48 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.434975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8590f3-d64e-4190-b2f7-f2f297b18c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.463013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-dd8590f3-d64e-4190-b2f7-f2f297b18c48 None None] MFA Rules not processed for user `db1a21e1056443ae9317c997c25d69c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:07.464245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dd8590f3-d64e-4190-b2f7-f2f297b18c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:07.526772 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1122/2231] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:07.537473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:07.539118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:07.550170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:07.616299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59690e-fb4f-409b-805f-c5aa47e770a4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:07.621970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:07.622591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:07.622873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:07.623157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:07.631413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:07.633055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:07.642063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1110/2232] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:09.337943 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f95dba-4caa-43f3-a55a-258c30619406 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:09.338430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f95dba-4caa-43f3-a55a-258c30619406 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:09.338520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f95dba-4caa-43f3-a55a-258c30619406 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:09.357022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a4f95dba-4caa-43f3-a55a-258c30619406 None None] MFA Rules not processed for user `d33866c1c77541a4bfd12d441e7ab175`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:09.358229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a4f95dba-4caa-43f3-a55a-258c30619406 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:09.378970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:09.417764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1123/2233] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:11.239158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:11.240619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:11.308061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd46f560-c130-4a02-aa8c-50b5d80d267e service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:11.312640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:11.313284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:11.313562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:11.313836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] PATH_INFO: `/v3/users/016ac28e6ce34a5eb78c62b3a31fe4fc` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:11.315006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=016ac28e6ce34a5eb78c62b3a31fe4fc)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:11.316986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:11.366853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '016ac28e6ce34a5eb78c62b3a31fe4fc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.380391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '016ac28e6ce34a5eb78c62b3a31fe4fc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.381127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '016ac28e6ce34a5eb78c62b3a31fe4fc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.381787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '016ac28e6ce34a5eb78c62b3a31fe4fc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.390695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '016ac28e6ce34a5eb78c62b3a31fe4fc'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.411023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1111/2234] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:11 2022] DELETE /identity/v3/users/016ac28e6ce34a5eb78c62b3a31fe4fc => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:11.415494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:11.415962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:11.416055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:11.434871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:11.436106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:11.494707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1124/2235] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:11.504246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:11.505646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:11.520428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:11.565089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58cd73c-af7c-48bc-8569-8b547787dc06 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:11.568868 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:11.569488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:11.570325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:11.570603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:11.583261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:11.584726 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e63242de-2701-4e82-9a0b-0d188c7054e5 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:11.594012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1112/2236] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:11.863131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:11.865439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:11.878707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c4624-a978-4046-9fa0-16f2db064581 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:11.885045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:11.885982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:11.886419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:11.886863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] PATH_INFO: `/v3/projects/b8ca34831b91428fbc3af1303c9be786` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:11.888117 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8ca34831b91428fbc3af1303c9be786)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:11.890048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:11.942509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8ca34831b91428fbc3af1303c9be786', 'request_id': 'req-a1985aa2-0d06-4936-9050-45082f071db6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.950799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8ca34831b91428fbc3af1303c9be786', 'request_id': 'req-a1985aa2-0d06-4936-9050-45082f071db6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.957311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8ca34831b91428fbc3af1303c9be786', 'request_id': 'req-a1985aa2-0d06-4936-9050-45082f071db6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.969149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] The token cache is being invalidate because project b8ca34831b91428fbc3af1303c9be786 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:11.969435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.970083 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1985aa2-0d06-4936-9050-45082f071db6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:11.971617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1125/2237] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:11 2022] DELETE /identity/v3/projects/b8ca34831b91428fbc3af1303c9be786 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:11.976907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:11.977184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:11.977454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:11.996767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:11.997938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:12.054775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1113/2238] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:12.062774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.063009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.063317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.082801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:12.084036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:12.092000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1126/2239] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:12.096171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.108876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0625c92f-f3bc-4ea9-b66d-0badcce649af None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.113980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.114593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.114875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.115149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.115904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:12.116212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.118555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.130300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1114/2240] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:12.134626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.146397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969f9ae3-31ae-40f7-9193-9e8cb23f5f14 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.151188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.151845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.152122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.152392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.153078 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.154967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.185531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1127/2241] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:12 2022] POST /identity/v3/projects => generated 373 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:12.189746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.199936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a70609-3502-46b1-9d3b-d2596f780f9c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.257007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.257730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.257903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.258181 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.258856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.260807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.285627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1115/2242] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:12 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:12.286797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.297747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cf1e62-12e8-4fe3-962b-12a1167e83a7 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.302580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.303251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.303570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.303891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.304586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.306221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.316125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1128/2243] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:12.320002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.330179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625df247-3cb8-4150-ba88-cd711cd4574b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.335038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.336220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.336502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.336779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.337478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.339066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaaea950-9e4a-49e3-ab60-32471872e52f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.349540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1116/2244] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:12.354314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.364229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d0826a-ab04-441e-8b59-e8b684a5a442 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.369400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.370051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.370333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.370611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] PATH_INFO: `/v3/projects/d9ff0edf56fa403e8b9bf25fe004cdd3/users/26ca8614358245b99f621edc7f1d0021/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.381709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:12.386813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ff0edf56fa403e8b9bf25fe004cdd3, user_id=26ca8614358245b99f621edc7f1d0021, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.389963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.402636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1129/2245] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:12 2022] PUT /identity/v3/projects/d9ff0edf56fa403e8b9bf25fe004cdd3/users/26ca8614358245b99f621edc7f1d0021/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:12.407816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.408105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.408382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.427507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:12.428711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:12.446355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:12.490182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1117/2246] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:12.500440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:12.502228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:12.557552 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a58a5b1-5270-40e2-b2aa-aeee611895a9 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:12.561272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:12.561880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:12.562160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:12.562432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:12.571931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:12.573287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d243743d-9c08-4eab-baf1-9b44d269fae2 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:12.582899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1130/2247] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:14.559765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:14.561546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:14.576767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c139a9e0-d425-4a36-ab9f-65e9c27a37dd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:14.582133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:14.582629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:14.582914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:14.583189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] PATH_INFO: `/v3/users/57aaf5bf00da496ab2ceb38609f4dc93` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:14.584499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57aaf5bf00da496ab2ceb38609f4dc93)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:14.586445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:14.775494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57aaf5bf00da496ab2ceb38609f4dc93'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:14.791745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57aaf5bf00da496ab2ceb38609f4dc93'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:14.792295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57aaf5bf00da496ab2ceb38609f4dc93'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:14.793661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57aaf5bf00da496ab2ceb38609f4dc93'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:14.805001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57aaf5bf00da496ab2ceb38609f4dc93'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:14.826211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1118/2248] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:14 2022] DELETE /identity/v3/users/57aaf5bf00da496ab2ceb38609f4dc93 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:14.828219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:14.828726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:14.829116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:14.851554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:14.853255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:14.943473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1131/2249] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:14.953891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:14.955371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:15.054663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e48e57-8b23-4fed-8575-753a636eb2a6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:15.058616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:15.059522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.059867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.060274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.070658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:15.072189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d118333-a8be-4e9d-88cd-f0de90fa0506 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:15.083392 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1119/2250] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:15.500871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:15.502886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:15.531087 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102cce0d-d2f3-42fe-8297-e546019be96e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:15.536474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:15.537135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.537419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.537697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] PATH_INFO: `/v3/projects/8df162614de9479e8327756f9324a19e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.541056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8df162614de9479e8327756f9324a19e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:15.543053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:15.609602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8df162614de9479e8327756f9324a19e', 'request_id': 'req-7c95679a-382b-431d-91ef-80fc4c7a2935'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:15.620788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8df162614de9479e8327756f9324a19e', 'request_id': 'req-7c95679a-382b-431d-91ef-80fc4c7a2935'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:15.629892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8df162614de9479e8327756f9324a19e', 'request_id': 'req-7c95679a-382b-431d-91ef-80fc4c7a2935'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:15.646650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] The token cache is being invalidate because project 8df162614de9479e8327756f9324a19e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:15.646968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:15.648671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7c95679a-382b-431d-91ef-80fc4c7a2935 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:15.655382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1132/2251] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:15 2022] DELETE /identity/v3/projects/8df162614de9479e8327756f9324a19e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:15.657145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.657626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.657708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.690860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:15.692767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:15.738130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:15.775072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1120/2252] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:15.845023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.845862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.846286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.867745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:15.869223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:15.882959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1133/2253] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:15.883854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:15.894397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9272dd4-346e-49d5-b79d-3e1633b489c3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:15.899875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:15.900531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.901104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.901379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.902108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:15.902414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:15.904072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:15.921803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1121/2254] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:15.923133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:15.933612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063a81f4-7174-4bcd-9fbd-28aff4d0415c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:15.939813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:15.940436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:15.940992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:15.941468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:15.942223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:15.944907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:15.951015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:15.980936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1134/2255] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:15 2022] POST /identity/v3/projects => generated 371 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:15.985342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:15.996945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6a42f4-a69b-46e2-a733-bc1cdb390cdb admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.075833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.077217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.077855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.078450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.079628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.081886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.109679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1122/2256] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:15 2022] POST /identity/v3/users => generated 286 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:16.113770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.135907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02aa774e-7a71-4e76-a05d-e07ed9e002c1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.153163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.153816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.154099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.154376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.155119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.156750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.171198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1135/2257] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:16.176890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.189658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc50f62f-0d67-4329-8fb1-5cddd4380295 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.193180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.196256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.196256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.196256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.196256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.197298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.200745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.202469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:16.209493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1123/2258] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:16.213572 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.218671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e0acfd-34a0-410c-b245-af0b5316e1bb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.226041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ab8af6-48a8-4137-ac5e-7193404b6c70 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.226577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.227266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.227598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.227882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] PATH_INFO: `/v3/users/db1a21e1056443ae9317c997c25d69c6` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.229071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db1a21e1056443ae9317c997c25d69c6)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.231190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.231295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.232188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.232535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.232875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] PATH_INFO: `/v3/projects/c7d258d1555247cb910985dd3a1b13b8/users/57021c6c6e0648dfaf3a85446e23576b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.250308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c7d258d1555247cb910985dd3a1b13b8, user_id=57021c6c6e0648dfaf3a85446e23576b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.253091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7bf601-1b54-485e-a11c-754f67e82165 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.271019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1124/2259] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:16 2022] PUT /identity/v3/projects/c7d258d1555247cb910985dd3a1b13b8/users/57021c6c6e0648dfaf3a85446e23576b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:16.281152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.281888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.281888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.291584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db1a21e1056443ae9317c997c25d69c6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:16.308007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db1a21e1056443ae9317c997c25d69c6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:16.308740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db1a21e1056443ae9317c997c25d69c6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:16.314968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] MFA Rules not processed for user `57021c6c6e0648dfaf3a85446e23576b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:16.316462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:16.318808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db1a21e1056443ae9317c997c25d69c6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:16.320546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2c808106-6848-4ed9-82db-48d28ad0c784 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db1a21e1056443ae9317c997c25d69c6'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:16.334380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1136/2260] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:16 2022] DELETE /identity/v3/users/db1a21e1056443ae9317c997c25d69c6 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:16.338885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.339188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.339781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.361609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:16.364160 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:16.404242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1125/2261] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:16 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:16.416634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.418018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:16.432712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:16.440266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1137/2262] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:16.451386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:16.453033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:16.507952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b814a40-5670-4066-a60c-98c76e8f6c56 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.515231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:16.520139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833d597-d72e-4cb4-9854-0f764e92bf22 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:16.526563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.527324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.527704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.527988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.553507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.555895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.567462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1138/2263] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:16.575279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:16.575953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:16.576228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:16.576495 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:16.671343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:16.673068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b12c48d-34eb-43fb-9ea7-082f22cfb79c service glance] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:16.700924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1126/2264] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:16 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 285 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:24:17.041171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.045081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:17.072536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c7e5d3-2604-46c0-ae99-e26bd1c25f99 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.088527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.089770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.090219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.091863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] PATH_INFO: `/v3/projects/976675956dca4c50917939c4d660e0b1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.097007 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] RBAC: Authorizing `identity:delete_project(project_id=976675956dca4c50917939c4d660e0b1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.098964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.226892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '976675956dca4c50917939c4d660e0b1', 'request_id': 'req-fa50b01b-6205-4e73-a852-b5f2147f8995'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:17.243564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '976675956dca4c50917939c4d660e0b1', 'request_id': 'req-fa50b01b-6205-4e73-a852-b5f2147f8995'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:17.253186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '976675956dca4c50917939c4d660e0b1', 'request_id': 'req-fa50b01b-6205-4e73-a852-b5f2147f8995'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:17.267079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] The token cache is being invalidate because project 976675956dca4c50917939c4d660e0b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:17.267950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:17.268893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fa50b01b-6205-4e73-a852-b5f2147f8995 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:17.270884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1139/2265] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:17 2022] DELETE /identity/v3/projects/976675956dca4c50917939c4d660e0b1 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:17.276513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.277469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.277980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.316343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:17.317661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:17.419527 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:17.440698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1127/2266] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:17.447517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.447736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.448019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.501177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:17.503096 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:17.530102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1140/2267] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:17.534257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.545323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d940c65-34ee-455f-bca5-415aebc31930 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.553197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.554542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.555414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.555584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.557201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:17.558580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.560314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.576372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1128/2268] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:17.581489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.595033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fb8891-2baf-4813-904b-e36cf31014d6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.601886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.602548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.602889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.603216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.603885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.605904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.648866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1141/2269] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:17 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:17.655152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.669617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c659a82-ed86-48ad-aef4-71e212308fd1 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.734237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.734237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.734237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.734237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.735126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.736516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.765742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1129/2270] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:17 2022] POST /identity/v3/users => generated 293 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:17.772224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.783874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8962498f-7a92-4372-99fc-45849cc71448 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.789145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.789776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.790058 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.790331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.791036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.792727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.812824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1142/2271] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:17.814132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.835233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf77fbee-9065-4be9-b8c2-f21bb586c6ab admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.848860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.851376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.851858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.852135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.853006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.862959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdfebf3-3420-4ec3-96bc-c82dc3c9acd3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.892195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1130/2272] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:17.896426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:17.909414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de002d2-e20b-4804-baa0-095748abcd8a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:17.915913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:17.916751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.917147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.917560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] PATH_INFO: `/v3/projects/49d2e9f00b2444cdbc00cbcccb77748e/users/ffccb066a95d467f86d6f2aa7563f42f/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:17.936436 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49d2e9f00b2444cdbc00cbcccb77748e, user_id=ffccb066a95d467f86d6f2aa7563f42f, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:17.939599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:17.957348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1143/2273] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:17 2022] PUT /identity/v3/projects/49d2e9f00b2444cdbc00cbcccb77748e/users/ffccb066a95d467f86d6f2aa7563f42f/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:17.968595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:17.968883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:17.969338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.015254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] MFA Rules not processed for user `ffccb066a95d467f86d6f2aa7563f42f`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:18.017341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:18.194954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1131/2274] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:18.199050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:18.318466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1035e7-3615-4a85-9e67-510c835f44f0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:18.328949 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:18.329610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.329891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.330262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.330857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:18.334252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:18.358625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:18.383688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1144/2275] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:18 2022] POST /identity/v3/projects => generated 387 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:18.391132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:18.412210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d3afbc-d300-4e0d-b2a1-331bc0b92018 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:18.511850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:18.525581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:18.527245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.531633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.532123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.536219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:18.538431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:18.621983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1132/2276] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:18 2022] POST /identity/v3/users => generated 294 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:18.626233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:18.646427 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe80d6-ca1a-4942-bab7-9885d7b69e04 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:18.686079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:18.686283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.686511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.686789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.687537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:18.689332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:18.708322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1145/2277] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:18.713082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:18.725292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21591feb-b9b2-474b-8389-37c981d39e44 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:18.737369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:18.738355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.738747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.739176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.740057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:18.741932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1b61-84a9-4485-aaee-47d6f1bb45d4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:18.755768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1133/2278] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:18.761860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:18.773335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f551ca-8d99-4c0a-a3bf-42368a865762 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:18.782294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:18.783317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.783840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.784329 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] PATH_INFO: `/v3/projects/b016d452964b4297a6ed74458a561184/users/c5524bee930047928421666b2757f327/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.812065 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b016d452964b4297a6ed74458a561184, user_id=c5524bee930047928421666b2757f327, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:18.817862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:18.845412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1146/2279] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:18 2022] PUT /identity/v3/projects/b016d452964b4297a6ed74458a561184/users/c5524bee930047928421666b2757f327/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:18.856540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:18.856830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:18.857127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:18.896761 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] MFA Rules not processed for user `c5524bee930047928421666b2757f327`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:18.898953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:18.999216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1134/2280] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:19.003741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.084227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0a3bc-71e1-400f-91aa-c1552b1b2e71 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.089825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.090557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.090912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.091272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.092130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.094325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.145113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1147/2281] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:19 2022] POST /identity/v3/projects => generated 385 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:19.145303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.163910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40aab5b-c94f-495d-bdae-696e01f482b2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.256918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.258259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.258472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.258756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.259505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.262125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.305466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1135/2282] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:19 2022] POST /identity/v3/users => generated 299 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:19.307177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.320018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5be2c5a-8164-40bf-97b6-9660b8cc0cd1 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.325908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.328357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.328898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.329326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.330731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.332545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.359512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.359702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1148/2283] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:19.382754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb178d5-388f-4312-a8bc-0a792b916031 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.391330 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.392083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.392361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.392633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] PATH_INFO: `/v3/projects/c42240ae47d047c7abccc8fc379c0120/users/d73d47d227bb42a88f25e08df79af129/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.451879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42240ae47d047c7abccc8fc379c0120, user_id=d73d47d227bb42a88f25e08df79af129, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.454852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.471913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1136/2284] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:19 2022] PUT /identity/v3/projects/c42240ae47d047c7abccc8fc379c0120/users/d73d47d227bb42a88f25e08df79af129/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:19.476252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.487945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28cf190-f9a5-498a-b740-b221fc5b9d15 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.559584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.560314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.560681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.561008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.561785 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.563805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fc82f7-d487-44e7-925d-ba872618664c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.579232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1149/2285] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:19.580493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.592193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16254543-420b-4715-a379-eb0b639e8cc3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.597582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.598270 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.598550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.598821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] PATH_INFO: `/v3/projects/c42240ae47d047c7abccc8fc379c0120/users/d73d47d227bb42a88f25e08df79af129/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.600930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] RBAC: Authorizing `identity:create_grant(project_id=c42240ae47d047c7abccc8fc379c0120, user_id=d73d47d227bb42a88f25e08df79af129, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.603801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.614930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1137/2286] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:19 2022] PUT /identity/v3/projects/c42240ae47d047c7abccc8fc379c0120/users/d73d47d227bb42a88f25e08df79af129/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:19.625473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb59de-b14d-4a21-973e-25fe06afe700 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.625840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb59de-b14d-4a21-973e-25fe06afe700 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.626198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02eb59de-b14d-4a21-973e-25fe06afe700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.657271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-02eb59de-b14d-4a21-973e-25fe06afe700 None None] MFA Rules not processed for user `d73d47d227bb42a88f25e08df79af129`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:19.658377 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-02eb59de-b14d-4a21-973e-25fe06afe700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:19.684270 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:19.821908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:19.822472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1150/2287] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:19 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:19.824117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:19.860733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:19.923359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740c76b9-2b26-4e5f-97e8-861310cc345b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:19.929215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:19.929964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:19.931926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:19.932280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:19.951287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:19.953191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:19.966017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1138/2288] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:19 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:20.118806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.119097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.119379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.171712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] MFA Rules not processed for user `26ca8614358245b99f621edc7f1d0021`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:20.173015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:20.279759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1151/2289] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:20 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:20.284323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.285779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:20.360048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0779185-c4a3-465c-986f-a2b7ff6f8172 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.364550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.365164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.365439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.365734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.367096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.369021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.406581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1139/2290] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:20 2022] POST /identity/v3/projects => generated 373 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:20.408081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.419405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1913ae3c-925e-4e0c-93ca-bb7ee22ba149 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.498380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.499566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.499931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.500274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.501047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.503199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.540656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1152/2291] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:20 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:20.544333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.558472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8910401-d880-4287-bd8b-854656484d51 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.568120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.568774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.569085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.570080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.571225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.573445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.588566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1140/2292] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:20.600262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.611124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0cca701-c5f8-44dc-83a1-86e96d7fe719 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.617373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.618107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.618390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.618672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] PATH_INFO: `/v3/projects/a062733c92a64fd68acb9a59fbba8b24/users/2a8c741e516c43b1844320c9f8b0e7ff/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.638823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a062733c92a64fd68acb9a59fbba8b24, user_id=2a8c741e516c43b1844320c9f8b0e7ff, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.643515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.665303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1153/2293] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:20 2022] PUT /identity/v3/projects/a062733c92a64fd68acb9a59fbba8b24/users/2a8c741e516c43b1844320c9f8b0e7ff/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:20.667067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.677559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe5f150-6b43-4e70-9fe2-8b8e44be7bc8 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.688059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.702534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:20.755871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.758128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.758128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.758394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.759047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.764400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.769771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27d1c30-0845-4310-8464-aaf10f73ca31 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.778755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1141/2294] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:20.780180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:20.782959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.783859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.785320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.785320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.800741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f80ae2-d0bc-4b0c-83fd-3bbe960dea36 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:20.814574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:20.815966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:20.816243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:20.816516 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] PATH_INFO: `/v3/projects/a062733c92a64fd68acb9a59fbba8b24/users/2a8c741e516c43b1844320c9f8b0e7ff/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:20.821379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a062733c92a64fd68acb9a59fbba8b24, user_id=2a8c741e516c43b1844320c9f8b0e7ff, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.825418 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.846771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1142/2295] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:20 2022] PUT /identity/v3/projects/a062733c92a64fd68acb9a59fbba8b24/users/2a8c741e516c43b1844320c9f8b0e7ff/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:20.897650 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:20.900059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d9983a-cf44-4909-835d-c0540c7060e6 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:20.911821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1154/2296] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:20 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:24:25.599324 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:25.619947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:25.679906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e890c4d1-66fa-4ae3-b4cc-7ea30a598002 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:25.689829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:25.690860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:25.691314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:25.691767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:25.767350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:25.768837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:25.780965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1143/2297] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:25 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:26.625141 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337718-5715-4b6a-9734-e8bea2ccc748 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:26.626690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337718-5715-4b6a-9734-e8bea2ccc748 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:26.626975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da337718-5715-4b6a-9734-e8bea2ccc748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:26.718616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-da337718-5715-4b6a-9734-e8bea2ccc748 None None] MFA Rules not processed for user `2a8c741e516c43b1844320c9f8b0e7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:26.721069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-da337718-5715-4b6a-9734-e8bea2ccc748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:26.866407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1155/2298] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:26 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:26.874090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:26.876301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:26.989336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62f98102-46de-41de-9bad-cbbc011a9539 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:26.989336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:26.989336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:26.989336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:26.989723 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:27.057334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:27.059795 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:27.085385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1144/2299] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:26 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:28.255343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b383d7-89ac-44d9-becc-57f61ed01cb5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:28.256310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b383d7-89ac-44d9-becc-57f61ed01cb5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:28.257259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b383d7-89ac-44d9-becc-57f61ed01cb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:28.277408 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-42b383d7-89ac-44d9-becc-57f61ed01cb5 None None] MFA Rules not processed for user `26ca8614358245b99f621edc7f1d0021`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:28.278631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-42b383d7-89ac-44d9-becc-57f61ed01cb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:28.285583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1156/2300] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:28.510629 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:28.520879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:28.588271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1e946-1829-4898-a448-4b24206ae884 service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:28.593441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:28.594357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:28.594859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:28.595175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] PATH_INFO: `/v3/users/ffccb066a95d467f86d6f2aa7563f42f` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:28.596653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffccb066a95d467f86d6f2aa7563f42f)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:28.598625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:28.694701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffccb066a95d467f86d6f2aa7563f42f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:28.710551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffccb066a95d467f86d6f2aa7563f42f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:28.711369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffccb066a95d467f86d6f2aa7563f42f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:28.712209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffccb066a95d467f86d6f2aa7563f42f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:28.723137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffccb066a95d467f86d6f2aa7563f42f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:28.749702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1145/2301] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:28 2022] DELETE /identity/v3/users/ffccb066a95d467f86d6f2aa7563f42f => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:28.754712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:28.755233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:28.755664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:28.779615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:28.782605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:28.844706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:28.875047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1157/2302] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:28.882496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:28.883985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:28.942333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dd65df-66c9-43ce-8f86-6de4a92f55af admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:28.946090 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:28.946709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:28.946992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:28.947263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:28.957072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:28.958488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:28.968320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1146/2303] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:29.363104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:29.365558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:29.378519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbc615d-4908-402d-a87b-790b3f75dd58 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:29.391004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:29.391004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:29.391004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:29.391004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] PATH_INFO: `/v3/projects/49d2e9f00b2444cdbc00cbcccb77748e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:29.391452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49d2e9f00b2444cdbc00cbcccb77748e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:29.391452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:29.466770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49d2e9f00b2444cdbc00cbcccb77748e', 'request_id': 'req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.478488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49d2e9f00b2444cdbc00cbcccb77748e', 'request_id': 'req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.489776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49d2e9f00b2444cdbc00cbcccb77748e', 'request_id': 'req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.505108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] The token cache is being invalidate because project 49d2e9f00b2444cdbc00cbcccb77748e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:29.505407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.506073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.508358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1158/2304] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:29 2022] DELETE /identity/v3/projects/49d2e9f00b2444cdbc00cbcccb77748e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:29.513157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:29.526550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8decfc-dd29-42db-8bb4-c878367edc00 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:29.561821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:29.595144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:29.596068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:29.596349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:29.596620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] PATH_INFO: `/v3/users/c5524bee930047928421666b2757f327` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:29.597790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5524bee930047928421666b2757f327)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:29.599728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:29.651818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5524bee930047928421666b2757f327'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.665513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5524bee930047928421666b2757f327'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.666298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5524bee930047928421666b2757f327'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.666978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5524bee930047928421666b2757f327'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.674180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5524bee930047928421666b2757f327'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:29.693172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1147/2305] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:29 2022] DELETE /identity/v3/users/c5524bee930047928421666b2757f327 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:30.073155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:30.075031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:30.090001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa3acdc-20ba-49a4-b4f2-d76ec8776a52 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:30.165217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:30.165948 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:30.166233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:30.166508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] PATH_INFO: `/v3/projects/b016d452964b4297a6ed74458a561184` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:30.167877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b016d452964b4297a6ed74458a561184)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:30.169976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:30.232341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b016d452964b4297a6ed74458a561184', 'request_id': 'req-3ed3f725-192b-4661-b8aa-1350d35a7633'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.243305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b016d452964b4297a6ed74458a561184', 'request_id': 'req-3ed3f725-192b-4661-b8aa-1350d35a7633'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.252290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b016d452964b4297a6ed74458a561184', 'request_id': 'req-3ed3f725-192b-4661-b8aa-1350d35a7633'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.260968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:30.269079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] The token cache is being invalidate because project b016d452964b4297a6ed74458a561184 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:30.269647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.270771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.273415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1159/2306] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:30 2022] DELETE /identity/v3/projects/b016d452964b4297a6ed74458a561184 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:30.277160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:30.278678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:30.293352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3942c300-7677-41e8-80f5-d014b9433d14 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:30.356505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:30.357098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:30.357383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:30.357655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] PATH_INFO: `/v3/users/d73d47d227bb42a88f25e08df79af129` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:30.358861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d73d47d227bb42a88f25e08df79af129)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:30.360802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:30.419917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd73d47d227bb42a88f25e08df79af129'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.421838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:30.435610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd73d47d227bb42a88f25e08df79af129'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.436551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd73d47d227bb42a88f25e08df79af129'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.437186 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd73d47d227bb42a88f25e08df79af129'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.445815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-2868b4f7-0676-4626-97aa-a6fac2155a00 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd73d47d227bb42a88f25e08df79af129'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.463571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1148/2307] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:30 2022] DELETE /identity/v3/users/d73d47d227bb42a88f25e08df79af129 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:30.772850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:30.775222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:30.794758 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed3f725-192b-4661-b8aa-1350d35a7633 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:30.869516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:30.870190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:30.870472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:30.870926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] PATH_INFO: `/v3/projects/c42240ae47d047c7abccc8fc379c0120` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:30.872415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c42240ae47d047c7abccc8fc379c0120)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:30.874654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:30.932889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c42240ae47d047c7abccc8fc379c0120', 'request_id': 'req-d0783851-3ec7-48e6-b7be-d2e6425d8efc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.948922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c42240ae47d047c7abccc8fc379c0120', 'request_id': 'req-d0783851-3ec7-48e6-b7be-d2e6425d8efc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.959250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c42240ae47d047c7abccc8fc379c0120', 'request_id': 'req-d0783851-3ec7-48e6-b7be-d2e6425d8efc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.974904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] The token cache is being invalidate because project c42240ae47d047c7abccc8fc379c0120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:30.975869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.976602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d0783851-3ec7-48e6-b7be-d2e6425d8efc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:30.978866 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1160/2308] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:30 2022] DELETE /identity/v3/projects/c42240ae47d047c7abccc8fc379c0120 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:30.985636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:30.986169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:30.986385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.011880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:31.013387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:31.105544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1149/2309] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:31.114149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.114440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.114720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.145157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:31.146425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:31.163456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1161/2310] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:31.167839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.177619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e50c68-1c45-471d-9076-ef77e731043e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.183124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.183771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.184049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.184319 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.185162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:31.185466 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.187041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.203023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1150/2311] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:31.206701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.218717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e3372d-46e9-43b8-b7e8-878f9c052a28 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.223913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.224540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.224818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.225119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.225987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.228443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.278418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1162/2312] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/projects => generated 401 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:31.279957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.295837 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db2935c-b4a9-4172-9ea9-54271581eea2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.371204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.371948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.372308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.372658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.374027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.375898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.402984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1151/2313] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/users => generated 301 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:31.404487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.418212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2a714a-6557-4391-9bd5-eebae2692dc8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.423742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.424524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.424808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.425085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.425795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.427517 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.442298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1163/2314] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:31.443456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.454898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00fb8ed7-a60b-4363-b1c9-949c77ab8da0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.461886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.462681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.462991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.463269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.464109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.468277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c0f4f-924e-4940-95e8-5e504d10d1b9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.483560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1152/2315] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:31.485384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.499648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-227c126c-da04-4f1c-aa60-a13bb0fbe306 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.504886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.505569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.505853 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.506131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] PATH_INFO: `/v3/projects/426d33d7a3a64ab1a312f8654be50f0e/users/d66c7f1f6a0549f0ad4886be71c57995/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.523798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=426d33d7a3a64ab1a312f8654be50f0e, user_id=d66c7f1f6a0549f0ad4886be71c57995, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.526560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.546213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1164/2316] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:31 2022] PUT /identity/v3/projects/426d33d7a3a64ab1a312f8654be50f0e/users/d66c7f1f6a0549f0ad4886be71c57995/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:31.560238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663ae0-119c-45ac-8456-b538842459bb None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.560522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663ae0-119c-45ac-8456-b538842459bb None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.560844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21663ae0-119c-45ac-8456-b538842459bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.592326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-21663ae0-119c-45ac-8456-b538842459bb None None] MFA Rules not processed for user `d66c7f1f6a0549f0ad4886be71c57995`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:31.593547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-21663ae0-119c-45ac-8456-b538842459bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:31.617834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:31.669710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1153/2317] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:31.674493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.708120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:31.753887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32759df9-fa3e-4e1a-87f1-291cd065fd55 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.761097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.761796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.762080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.762355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.763046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.766388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.802045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1165/2318] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:31.807715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663ae0-119c-45ac-8456-b538842459bb None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.822909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21663ae0-119c-45ac-8456-b538842459bb None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.935721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.936364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.936644 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.936921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.937601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.942595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:31.973344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1154/2319] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:31 2022] POST /identity/v3/users => generated 307 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:31.974670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:31.986925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5531a070-d2a7-4a6e-838e-6c90464a9dea admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:31.992698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:31.993337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:31.993622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:31.993897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:31.994607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:31.996686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.014201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1166/2320] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:31 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:32.019403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:32.036902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b446ea8f-5307-4ab4-9145-2e13cb14b49b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:32.050899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:32.051805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.052067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.052393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] PATH_INFO: `/v3/projects/42957535933041618cf37f8cd4fd7eea/users/8a65547c96ed4ede95a09b067eaefc11/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.073386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42957535933041618cf37f8cd4fd7eea, user_id=8a65547c96ed4ede95a09b067eaefc11, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:32.076697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.100602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1155/2321] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:32 2022] PUT /identity/v3/projects/42957535933041618cf37f8cd4fd7eea/users/8a65547c96ed4ede95a09b067eaefc11/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:32.106056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:32.127213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac90fcd4-ab80-49cd-a053-a605e7e76b99 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:32.231475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:32.232114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.232401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.232682 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.233390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:32.236562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d66f8e-6c5d-4ab7-96ab-486c49e5eb34 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.249752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1167/2322] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:32 2022] GET /identity/v3/roles => generated 1420 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:32.254756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:32.268852 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b54502-30e6-4e19-b142-64c0d8bb7877 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:32.277911 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:32.278702 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.280036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.280429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] PATH_INFO: `/v3/projects/42957535933041618cf37f8cd4fd7eea/users/8a65547c96ed4ede95a09b067eaefc11/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.286223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] RBAC: Authorizing `identity:create_grant(project_id=42957535933041618cf37f8cd4fd7eea, user_id=8a65547c96ed4ede95a09b067eaefc11, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:32.290298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.306712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1156/2323] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:32 2022] PUT /identity/v3/projects/42957535933041618cf37f8cd4fd7eea/users/8a65547c96ed4ede95a09b067eaefc11/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:32.316179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.316471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.316747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.362365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] MFA Rules not processed for user `8a65547c96ed4ede95a09b067eaefc11`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:32.363637 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:32.471514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1168/2324] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:32 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:32.483139 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:32.484963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:32.560834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb030ccf-598c-4af3-924c-8b535cefe152 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:32.565498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:32.565498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.565786 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.566035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.589542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:32.591029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.618799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1157/2325] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:32.732008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:32.733601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:32.751567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29294dc-431b-489f-ad3d-a67fe4abd596 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:32.755334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:32.756313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:32.756686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:32.756962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:32.779248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:32.828721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:32.830153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc9cd10e-0c77-4608-a3b5-e61e2943c89b service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:32.840001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1169/2326] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:40.369481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:40.388570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:40.432023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3440d99e-38de-40ef-9ad5-22a4985e1cb4 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:40.436556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:40.437189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:40.437468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:40.437734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] PATH_INFO: `/v3/users/d66c7f1f6a0549f0ad4886be71c57995` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:40.438909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d66c7f1f6a0549f0ad4886be71c57995)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:40.440833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:40.485375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd66c7f1f6a0549f0ad4886be71c57995'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:40.497711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd66c7f1f6a0549f0ad4886be71c57995'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:40.498397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd66c7f1f6a0549f0ad4886be71c57995'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:40.498997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd66c7f1f6a0549f0ad4886be71c57995'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:40.508106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd66c7f1f6a0549f0ad4886be71c57995'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:40.524016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1158/2327] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:40 2022] DELETE /identity/v3/users/d66c7f1f6a0549f0ad4886be71c57995 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:40.528462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:40.528910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:40.529010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:40.550654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:40.551904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:40.612746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1170/2328] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:40.628496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:40.629971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:40.693011 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eba4353-d941-44e4-9537-ffb2af94a585 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:40.696574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:40.697187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:40.697492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:40.697740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:40.706621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:40.708077 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:40.717748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1159/2329] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:41.017231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:41.019076 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.033810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba0e794-2c58-4c87-9851-72ed3ae124a7 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:41.039298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:41.039980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.040267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.040605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] PATH_INFO: `/v3/projects/426d33d7a3a64ab1a312f8654be50f0e` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.041712 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=426d33d7a3a64ab1a312f8654be50f0e)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:41.043662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:41.097111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '426d33d7a3a64ab1a312f8654be50f0e', 'request_id': 'req-0ffcf5e1-436e-4190-8bda-186da6130f3d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.105015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '426d33d7a3a64ab1a312f8654be50f0e', 'request_id': 'req-0ffcf5e1-436e-4190-8bda-186da6130f3d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.111690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '426d33d7a3a64ab1a312f8654be50f0e', 'request_id': 'req-0ffcf5e1-436e-4190-8bda-186da6130f3d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.121937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] The token cache is being invalidate because project 426d33d7a3a64ab1a312f8654be50f0e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:41.122222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.122869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-0ffcf5e1-436e-4190-8bda-186da6130f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.124415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1171/2330] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:41 2022] DELETE /identity/v3/projects/426d33d7a3a64ab1a312f8654be50f0e => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:41.128431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:41.138662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29e5a61-1f9f-4ddd-8780-fb52acfbc053 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:41.176278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:41.197267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:41.197909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.198187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.198469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] PATH_INFO: `/v3/users/8a65547c96ed4ede95a09b067eaefc11` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.199690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8a65547c96ed4ede95a09b067eaefc11)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:41.201822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:41.245405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8a65547c96ed4ede95a09b067eaefc11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.257531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8a65547c96ed4ede95a09b067eaefc11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.258183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8a65547c96ed4ede95a09b067eaefc11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.258772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8a65547c96ed4ede95a09b067eaefc11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.265558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8a65547c96ed4ede95a09b067eaefc11'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.279343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1160/2331] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:41 2022] DELETE /identity/v3/users/8a65547c96ed4ede95a09b067eaefc11 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:41.358198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.358578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.358922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.380445 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:41.382204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.424693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:41.451927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1172/2332] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:24:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:41.493531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:41.494950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.579581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91170753-2891-4cd9-a44a-934e59015021 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:41.586304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:41.587001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.587210 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.587525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.593386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:41.594903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.601985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:41.603993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c73f669-22f4-4f54-b074-393637cb1d96 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:41.615557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1161/2333] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:24:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:24:41.692554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841a2d76-ad22-4dc0-824f-9668798e9b70 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:41.698673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:41.699331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.699638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.699917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] PATH_INFO: `/v3/projects/42957535933041618cf37f8cd4fd7eea` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.701344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] RBAC: Authorizing `identity:delete_project(project_id=42957535933041618cf37f8cd4fd7eea)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:41.703409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:41.754045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '42957535933041618cf37f8cd4fd7eea', 'request_id': 'req-e27b94a4-bec2-4532-9939-6ecb11691929'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.765191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '42957535933041618cf37f8cd4fd7eea', 'request_id': 'req-e27b94a4-bec2-4532-9939-6ecb11691929'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.774555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '42957535933041618cf37f8cd4fd7eea', 'request_id': 'req-e27b94a4-bec2-4532-9939-6ecb11691929'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.790017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] The token cache is being invalidate because project 42957535933041618cf37f8cd4fd7eea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:41.790581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.792056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e27b94a4-bec2-4532-9939-6ecb11691929 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:41.794248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1173/2334] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:41 2022] DELETE /identity/v3/projects/42957535933041618cf37f8cd4fd7eea => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:41.799673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.800161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.800593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.824720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:41.829435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.884942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:41.907908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1162/2335] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:41.916419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.917004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.917474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.939514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:41.940756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:41.951090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1174/2336] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:41.956307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:41.969900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c22bbf2-595d-44ac-b7ef-297d27ca41ee None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:41.975910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:41.976800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:41.977249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:41.977697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:41.978571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:41.978892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:41.980551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:41.999592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1163/2337] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:42.006802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.019662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92abd9e1-6d81-409c-a36d-0760fd2459de None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.026622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.027875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.028276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.028487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.029360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.031415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.072475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1175/2338] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:42 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:42.073396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.086373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f35a64-f983-43c5-b938-c5f7e817d711 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.196148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.197291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.197575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.197850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.198925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.201648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.233764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1164/2339] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:42 2022] POST /identity/v3/users => generated 291 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:24:42.238363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.251929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3710a8ea-dd4b-49ce-93bb-6774d46fb629 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.258126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.258766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.259091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.259541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.260365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.263006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.282565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1176/2340] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:42.283720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.294870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14468094-c3a5-438a-83b8-09b011d91a70 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.301269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.302048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.302442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.302717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.304249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.306148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e251f-6d08-4b9a-a433-5e4d7a14c92b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.327305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1165/2341] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:24:42.328267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.341954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffdffab-fdaa-41b3-9d74-8b9967802fb5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.348356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.349520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.350038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.350364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] PATH_INFO: `/v3/projects/e5d8206d50a44a1ea1bc8306729f961d/users/8efc2a4144ee4e2dbe8e0dc2ee103d8d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.373995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5d8206d50a44a1ea1bc8306729f961d, user_id=8efc2a4144ee4e2dbe8e0dc2ee103d8d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.378038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.407892 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1177/2342] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:42 2022] PUT /identity/v3/projects/e5d8206d50a44a1ea1bc8306729f961d/users/8efc2a4144ee4e2dbe8e0dc2ee103d8d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:24:42.430332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.430332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.430332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.458431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] MFA Rules not processed for user `8efc2a4144ee4e2dbe8e0dc2ee103d8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:42.460043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:42.539725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1166/2343] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:42.544753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.560481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:42.639658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b8b058-7771-40b1-83dd-684225a964e8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.666894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.667686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.668660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.669036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.670315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.675165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.733996 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1178/2344] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:42 2022] POST /identity/v3/projects => generated 383 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:42.738480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.758551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bac950-e11b-4e02-b6aa-8103176692dd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.811220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:42.828892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.829519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.829798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.830070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.830734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.832748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.856986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1167/2345] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:42 2022] POST /identity/v3/users => generated 298 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:42.861294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.873714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eae48c1-a36d-4eb6-a497-d3f844ecfc24 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.878739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.879360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.879713 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.879990 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.880814 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.882756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.896176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1179/2346] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:42.904298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:42.918153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04f4886-c2bf-4588-8c6e-616e5b5950f2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:42.923517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:42.924191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:42.924465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:42.924828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] PATH_INFO: `/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea/users/aa59d9f357824d6e9955e231a5bbec53/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:42.943129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5fdd77650d47e2a1782b57daf3e3ea, user_id=aa59d9f357824d6e9955e231a5bbec53, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:42.946514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:42.967621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1168/2347] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:42 2022] PUT /identity/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea/users/aa59d9f357824d6e9955e231a5bbec53/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:42.975623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:43.011253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df427a82-1f3f-4ac6-adbd-4cf182df4cc3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:43.158345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:43.158985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:43.159306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:43.159618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:43.160364 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:43.163289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df159155-1274-4e2e-82a4-e761fe1fc468 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:43.182474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1180/2348] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:42 2022] GET /identity/v3/roles => generated 1420 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:43.186710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:43.197711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a4aa6d-f09b-4450-b67a-40120466fc8e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:43.202745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:43.203575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:43.203919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:43.204200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] PATH_INFO: `/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea/users/aa59d9f357824d6e9955e231a5bbec53/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:43.206299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee5fdd77650d47e2a1782b57daf3e3ea, user_id=aa59d9f357824d6e9955e231a5bbec53, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:43.209564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:43.221673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1169/2349] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:43 2022] PUT /identity/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea/users/aa59d9f357824d6e9955e231a5bbec53/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:43.233534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:43.233875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:43.234269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:43.276480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] MFA Rules not processed for user `aa59d9f357824d6e9955e231a5bbec53`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:43.277981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:43.369429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1181/2350] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:43 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:43.381711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:43.383405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:43.453448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9390b921-f170-43ce-ab71-62758c1254db admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:43.458130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:43.458823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:43.459844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:43.460123 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:43.470376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:43.471923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:43.482229 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1170/2351] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:43 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:43.592627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:43.603263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43763326-69bc-4433-8ba7-5af6640858bd None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:43.607760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:43.608545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:43.609488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:43.609770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:43.617497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:43.691250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:43.693267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b840a3-da8a-46b7-8488-804aed8bf2a9 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:43.707593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1182/2352] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:43 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:46.603223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:46.664315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06d5052e-c750-42ab-9730-51df1b0ae544 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:46.668767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:46.669400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:46.669682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:46.669950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] PATH_INFO: `/v3/users/8efc2a4144ee4e2dbe8e0dc2ee103d8d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:46.671109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8efc2a4144ee4e2dbe8e0dc2ee103d8d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:46.673065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:46.715922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:46.721514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8efc2a4144ee4e2dbe8e0dc2ee103d8d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:46.741753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8efc2a4144ee4e2dbe8e0dc2ee103d8d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:46.742707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8efc2a4144ee4e2dbe8e0dc2ee103d8d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:46.743285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8efc2a4144ee4e2dbe8e0dc2ee103d8d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:46.750368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8efc2a4144ee4e2dbe8e0dc2ee103d8d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:46.763614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1171/2353] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:46 2022] DELETE /identity/v3/users/8efc2a4144ee4e2dbe8e0dc2ee103d8d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:46.768124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:46.768545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:46.768868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:46.789027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:46.790244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:46.849018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1183/2354] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:46.859018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:46.860424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:46.932871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13ea890-2741-467a-9755-6401735f6ef6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:46.936473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:46.937088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:46.937367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:46.937640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:46.946544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:46.948109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:46.958246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1172/2355] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:47.221458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:47.222938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:47.237928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7ab858-16cf-42ac-adde-ce16f4246746 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:47.242773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:47.243475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:47.243760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:47.244036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] PATH_INFO: `/v3/projects/e5d8206d50a44a1ea1bc8306729f961d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:47.245228 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5d8206d50a44a1ea1bc8306729f961d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:47.247157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:47.296656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5d8206d50a44a1ea1bc8306729f961d', 'request_id': 'req-e0166f24-754c-491f-aec3-ed1db9672f0e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.304895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5d8206d50a44a1ea1bc8306729f961d', 'request_id': 'req-e0166f24-754c-491f-aec3-ed1db9672f0e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.310914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5d8206d50a44a1ea1bc8306729f961d', 'request_id': 'req-e0166f24-754c-491f-aec3-ed1db9672f0e'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.321575 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] The token cache is being invalidate because project e5d8206d50a44a1ea1bc8306729f961d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:47.321860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.322482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.324038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1184/2356] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:47 2022] DELETE /identity/v3/projects/e5d8206d50a44a1ea1bc8306729f961d => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:47.328431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:47.338814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584f5fd-e4e3-41e4-87c4-3b5aa7ae2978 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:47.392716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:47.393395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:47.393679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:47.393974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] PATH_INFO: `/v3/users/aa59d9f357824d6e9955e231a5bbec53` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:47.395158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa59d9f357824d6e9955e231a5bbec53)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:47.397705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:47.445899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa59d9f357824d6e9955e231a5bbec53'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.457684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:47.458966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa59d9f357824d6e9955e231a5bbec53'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.460298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa59d9f357824d6e9955e231a5bbec53'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.461223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa59d9f357824d6e9955e231a5bbec53'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.469880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-8943d4a6-72c5-4711-98a0-30b12cab1234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa59d9f357824d6e9955e231a5bbec53'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.482708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1173/2357] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:47 2022] DELETE /identity/v3/users/aa59d9f357824d6e9955e231a5bbec53 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:47.768393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:47.769835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:47.784305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0166f24-754c-491f-aec3-ed1db9672f0e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:47.814206 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:47.843947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:47.844587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:47.844870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:47.845150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] PATH_INFO: `/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:47.846299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee5fdd77650d47e2a1782b57daf3e3ea)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:47.849086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:47.896165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee5fdd77650d47e2a1782b57daf3e3ea', 'request_id': 'req-1f0cce09-6607-481c-9724-37ca2a514a74'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.904737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee5fdd77650d47e2a1782b57daf3e3ea', 'request_id': 'req-1f0cce09-6607-481c-9724-37ca2a514a74'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.911466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee5fdd77650d47e2a1782b57daf3e3ea', 'request_id': 'req-1f0cce09-6607-481c-9724-37ca2a514a74'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.926019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] The token cache is being invalidate because project ee5fdd77650d47e2a1782b57daf3e3ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:47.926521 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.927619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1f0cce09-6607-481c-9724-37ca2a514a74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:47.930143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1185/2358] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:47 2022] DELETE /identity/v3/projects/ee5fdd77650d47e2a1782b57daf3e3ea => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:47.935159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:47.935977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:47.935977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:47.955364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:47.956589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:48.025903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1174/2359] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:48.034802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.034942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.035183 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.056392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:48.057606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:48.065528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1186/2360] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:48.069872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.079528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dd0b7e-ca08-43f1-807f-fc7bb2ec7102 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.084021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.084624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.084904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.085176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.085871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:24:48.086171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.087785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.099099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1175/2361] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:24:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:24:48.103880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.113126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e5c83-649a-492a-a4b1-acbee57886c8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.118398 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.118998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.119279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.119592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.120264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.122167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.154378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1187/2362] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:24:48 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:48.155615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.165913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96949ebd-ee2d-4301-a66a-448999207363 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.221129 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.221737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.222018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.222295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.222950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.225477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.245760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1176/2363] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:24:48 2022] POST /identity/v3/users => generated 284 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:24:48.250333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.264844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bc67d5-5fc4-48ee-b85f-ab407600fb1b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.280207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.280872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.281151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.281420 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.282131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.284212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.299380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1188/2364] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:48.305162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.315317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a987002a-b92c-40f6-aa1b-e23019a405a3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.319936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.320548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.320829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.321102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.321800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.323437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9165351-bef6-466f-98a9-238860ef4199 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.334622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1177/2365] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:24:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:24:48.338769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:48.348776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d4f2fe-0d5c-4ecf-8456-52278ab73e85 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:48.353940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:48.354596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.354877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.355156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] PATH_INFO: `/v3/projects/ba436ca4f3194b18840899392541fe2c/users/f0cc5dd7dbc142deb5f8813d149778ae/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.369634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] RBAC: Authorizing `identity:create_grant(project_id=ba436ca4f3194b18840899392541fe2c, user_id=f0cc5dd7dbc142deb5f8813d149778ae, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:48.372312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:48.385102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1189/2366] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:24:48 2022] PUT /identity/v3/projects/ba436ca4f3194b18840899392541fe2c/users/f0cc5dd7dbc142deb5f8813d149778ae/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:48.395464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:48.395750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:48.396023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:48.423694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] MFA Rules not processed for user `f0cc5dd7dbc142deb5f8813d149778ae`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:48.424906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:48.456425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:48.488200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1178/2367] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:24:51.282549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:51.284318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:51.356939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc6c6bc-5606-4445-bb27-e1fa619c931d admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:51.360554 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:51.361255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:51.361594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:51.361870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:51.371963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:51.373388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b40fd9-a416-4199-9f42-8132b139d1ae service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:51.378352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:51.384501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1190/2368] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:54.484037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:54.485785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:54.560817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a87da03-d4fd-41c8-819c-6e6cc9882a39 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:54.566165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:54.566832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:54.567178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:54.567560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] PATH_INFO: `/v3/users/57021c6c6e0648dfaf3a85446e23576b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:54.569436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] RBAC: Authorizing `identity:delete_user(user_id=57021c6c6e0648dfaf3a85446e23576b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:54.571496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:54.640068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '57021c6c6e0648dfaf3a85446e23576b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:54.655250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '57021c6c6e0648dfaf3a85446e23576b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:54.655989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '57021c6c6e0648dfaf3a85446e23576b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:54.656603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '57021c6c6e0648dfaf3a85446e23576b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:54.665228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '57021c6c6e0648dfaf3a85446e23576b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:54.681274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1179/2369] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:24:54 2022] DELETE /identity/v3/users/57021c6c6e0648dfaf3a85446e23576b => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:54.685946 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:54.686447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:54.686615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:54.705837 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:54.707033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:54.768027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1191/2370] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:24:54.778566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:54.780148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:54.839793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef38f4a2-d3ca-40da-a20e-fcfacde03182 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:54.843379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:54.844020 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:54.844301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:54.844574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:54.853880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:54.855302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf756eb-2d66-4173-a5e4-1079adb2b750 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:54.859168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:24:54.867560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1180/2371] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:24:55.249227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:24:55.251118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:55.300150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48136f8-1f36-4b49-a530-811c3dee7f2b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:24:55.322429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:24:55.323658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:55.324638 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:55.324993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] PATH_INFO: `/v3/projects/c7d258d1555247cb910985dd3a1b13b8` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:55.327071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c7d258d1555247cb910985dd3a1b13b8)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:24:55.330254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:24:55.446495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c7d258d1555247cb910985dd3a1b13b8', 'request_id': 'req-cda50a3b-c987-4b89-be26-aed48e016343'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:55.469826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c7d258d1555247cb910985dd3a1b13b8', 'request_id': 'req-cda50a3b-c987-4b89-be26-aed48e016343'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:55.488844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c7d258d1555247cb910985dd3a1b13b8', 'request_id': 'req-cda50a3b-c987-4b89-be26-aed48e016343'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:55.604832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] The token cache is being invalidate because project c7d258d1555247cb910985dd3a1b13b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:24:55.604832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:55.609260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-cda50a3b-c987-4b89-be26-aed48e016343 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:24:55.627589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1192/2372] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:24:55 2022] DELETE /identity/v3/projects/c7d258d1555247cb910985dd3a1b13b8 => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:24:57.040039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:24:57.040598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:24:57.040598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:24:57.060984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] MFA Rules not processed for user `d6e8399080a942f1b689fc8c6b42c717`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:24:57.062248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:24:57.139076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1181/2373] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:24:57 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:05.582211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:07.131159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=26 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:12.285251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:12.286241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:12.286525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:12.317388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:12.322826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:12.391174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1193/2374] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:25:12 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:12.405895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:12.408583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:12.485304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2de8f56c-a371-4999-9446-26ca7afeafc9 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:12.490276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:12.490882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:12.491153 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:12.491504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:12.499619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:12.501203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:12.509177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1182/2375] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:25:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:25:14.855688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:14.858034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:14.952407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5a55f8-b24d-46a9-b660-2c6d90c81e20 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:14.959221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:14.960775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:14.962134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:14.962134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] PATH_INFO: `/v3/users/d33866c1c77541a4bfd12d441e7ab175` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:14.963905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d33866c1c77541a4bfd12d441e7ab175)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:14.965927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.041767 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd33866c1c77541a4bfd12d441e7ab175'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.071964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd33866c1c77541a4bfd12d441e7ab175'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.072717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd33866c1c77541a4bfd12d441e7ab175'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.087225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd33866c1c77541a4bfd12d441e7ab175'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.087898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-89b024c4-210f-4975-b4c5-ce6a5e0660f5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd33866c1c77541a4bfd12d441e7ab175'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.123231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1194/2376] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:25:14 2022] DELETE /identity/v3/users/d33866c1c77541a4bfd12d441e7ab175 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:15.341176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.353614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d1b1d-b992-4624-b785-1f00d9624676 service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.427548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.428245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.428532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.428807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] PATH_INFO: `/v3/projects/b8656d0df1af4475b502efe677f7308e` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.429978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b8656d0df1af4475b502efe677f7308e)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.431932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.505355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b8656d0df1af4475b502efe677f7308e', 'request_id': 'req-95913bf8-6e7e-4117-931f-4b5774f4c289'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.513531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b8656d0df1af4475b502efe677f7308e', 'request_id': 'req-95913bf8-6e7e-4117-931f-4b5774f4c289'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.528107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b8656d0df1af4475b502efe677f7308e', 'request_id': 'req-95913bf8-6e7e-4117-931f-4b5774f4c289'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.541121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] The token cache is being invalidate because project b8656d0df1af4475b502efe677f7308e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:25:15.541416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.542057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-95913bf8-6e7e-4117-931f-4b5774f4c289 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:15.543616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1183/2377] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:25:15 2022] DELETE /identity/v3/projects/b8656d0df1af4475b502efe677f7308e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:25:15.548375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.548660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.548934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.570795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:15.572858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:15.594105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:15.644430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1195/2378] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:15.653582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.653971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.654253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.674027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:15.675344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:15.686805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1184/2379] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:15.688635 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.711616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5706ef06-bb40-4165-bacd-7718619ecd08 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.711616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.711616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.711616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.712305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.712305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:25:15.712305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.712305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.726284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1196/2380] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:25:15.730225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.739715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e090a9-63a6-44ae-90f7-76f8ad5be94b None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.744457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.745076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.745352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.745626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.746299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.748249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.761963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:15.800955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1185/2381] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:25:15 2022] POST /identity/v3/projects => generated 383 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:25:15.804770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.815602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf5b6f8-da22-4f13-9152-179cfa694baf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.894820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.895479 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.895843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.896152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.896931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.900349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.934523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.935464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1197/2382] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:25:15 2022] POST /identity/v3/users => generated 292 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:25:15.936593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:15.939115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.949895 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfb4951-6a7c-4c81-8bbd-6da81915d79f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.955052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.955815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.956196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.956578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.957373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.959069 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:15.971243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1198/2383] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:25:15.974414 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:15.984467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0115d99-ea58-4c82-bffe-6f9403f65611 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:15.992794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:15.993477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:15.993757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:15.994026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:15.994766 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:15.996386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:16.005175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286fd812-585c-412a-b4b1-55e04a6c4ce6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:16.008133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1199/2384] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:25:16.012332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:16.012965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:16.013665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:16.014034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:16.014388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:16.016262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:16.023453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b3035d-af41-4730-b058-e88ff79eeec3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:16.028752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:16.029417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:16.029694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:16.029968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] PATH_INFO: `/v3/projects/7d64c4c74e7e454d8e7ddb31f2a5075b/users/9853b94a14ab49a4b834d5a84cad9f4b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:16.050488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d64c4c74e7e454d8e7ddb31f2a5075b, user_id=9853b94a14ab49a4b834d5a84cad9f4b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:16.053883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be605508-c0f6-4569-9740-4bdb48568866 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:16.077484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1200/2385] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:25:16 2022] PUT /identity/v3/projects/7d64c4c74e7e454d8e7ddb31f2a5075b/users/9853b94a14ab49a4b834d5a84cad9f4b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:25:16.095503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:16.097075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:16.108160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1186/2386] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:15 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:16.152451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:16.152743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:16.153023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:16.184215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] MFA Rules not processed for user `9853b94a14ab49a4b834d5a84cad9f4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:16.185442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:16.233221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:16.258015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1201/2387] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:16 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:16.269822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:16.271199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:16.330756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a8cac7-de37-41d3-aeed-14d4d25beb4b service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:16.380522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:16.381265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:16.381951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:16.382223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:16.390771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:16.392234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7eb9a087-2cdc-4537-a0fb-9788e71e0e44 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:16.396768 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:16.403488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1187/2388] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:16 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:21.109627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:21.184986 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52d346b-cd2e-4d34-900c-567b5ea11761 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:21.189456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:21.190094 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:21.190373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:21.190651 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] PATH_INFO: `/v3/users/9853b94a14ab49a4b834d5a84cad9f4b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:21.191842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] RBAC: Authorizing `identity:delete_user(user_id=9853b94a14ab49a4b834d5a84cad9f4b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:21.193804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:21.238789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9853b94a14ab49a4b834d5a84cad9f4b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:21.250264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9853b94a14ab49a4b834d5a84cad9f4b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:21.250975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9853b94a14ab49a4b834d5a84cad9f4b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:21.257930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9853b94a14ab49a4b834d5a84cad9f4b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:21.258545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9853b94a14ab49a4b834d5a84cad9f4b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:21.270873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1202/2389] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:25:21 2022] DELETE /identity/v3/users/9853b94a14ab49a4b834d5a84cad9f4b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:21.275707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:21.276187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:21.276321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:21.294827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:21.296106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:21.352155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1188/2390] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:21.361588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:21.362967 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:21.421897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871f7526-949b-48a9-ac4f-05e99e28989b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:21.426075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:21.426698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:21.426980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:21.427255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:21.437323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:21.438904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848fc807-b69b-4d1c-b40d-746192aa9310 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:21.452691 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1203/2391] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:21.935198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:21.936926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:21.951810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f28f6a-e825-437d-87bb-0e03cd8b05a2 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:21.958541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:21.960143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:21.960656 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:21.960954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] PATH_INFO: `/v3/projects/7d64c4c74e7e454d8e7ddb31f2a5075b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:21.962360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d64c4c74e7e454d8e7ddb31f2a5075b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:21.964291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.019839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d64c4c74e7e454d8e7ddb31f2a5075b', 'request_id': 'req-308fb520-4908-4ae5-8049-30052d27ff91'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:22.027263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d64c4c74e7e454d8e7ddb31f2a5075b', 'request_id': 'req-308fb520-4908-4ae5-8049-30052d27ff91'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:22.035998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d64c4c74e7e454d8e7ddb31f2a5075b', 'request_id': 'req-308fb520-4908-4ae5-8049-30052d27ff91'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:22.047372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] The token cache is being invalidate because project 7d64c4c74e7e454d8e7ddb31f2a5075b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:25:22.047372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:22.047953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-308fb520-4908-4ae5-8049-30052d27ff91 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:22.049590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1189/2392] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:25:21 2022] DELETE /identity/v3/projects/7d64c4c74e7e454d8e7ddb31f2a5075b => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:22.054906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.055195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.055598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.070223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:22.077157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:22.079489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:22.140779 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1204/2393] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:22.148967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.149281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.149557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.170395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:22.171637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:22.180297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1190/2394] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:22.184841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.195761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9dcff9-93a2-450c-bed7-1f36e473641b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.200744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.201351 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.201634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.201906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.202696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:25:22.203809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.204665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.219456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1205/2395] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:25:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:25:22.223994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.234331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30626238-bbc4-47c4-a739-77fca6bd953f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.239505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.240232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.241146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.241518 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.242292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.244423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.278701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1191/2396] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:25:22 2022] POST /identity/v3/projects => generated 373 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:25:22.280489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.291670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50e06460-1565-4851-90e1-6a078426cd06 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.377488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.378186 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.378467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.378744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.379531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.382023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.416683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1206/2397] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:25:22 2022] POST /identity/v3/users => generated 287 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:25:22.417968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.433586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289c3ff7-7565-4e04-92ae-12750345be51 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.439870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.440853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.441430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.441904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.443013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.445061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.464174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1192/2398] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:25:22.472209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.484605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57f7727-d636-40f4-afd2-9dee78ec704e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.489678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.490362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.491806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.492048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.492852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.494418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c6f58-85cc-4f8b-811b-25aada56516e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.505645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1207/2399] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:22 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:25:22.509953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.522017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a510538-315e-4869-baf9-8031b5d662da admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.527008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.527863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.528145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.528461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] PATH_INFO: `/v3/projects/b0ad86716ddf412f91a1128ba7159585/users/b6d1952d79174b6596f829cd85ca690b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.543564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0ad86716ddf412f91a1128ba7159585, user_id=b6d1952d79174b6596f829cd85ca690b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.546232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.562214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1193/2400] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:25:22 2022] PUT /identity/v3/projects/b0ad86716ddf412f91a1128ba7159585/users/b6d1952d79174b6596f829cd85ca690b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:22.573915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564585-5efd-4cc7-913d-0b7b5b396781 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.574226 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564585-5efd-4cc7-913d-0b7b5b396781 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.574501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87564585-5efd-4cc7-913d-0b7b5b396781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.603786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-87564585-5efd-4cc7-913d-0b7b5b396781 None None] MFA Rules not processed for user `b6d1952d79174b6596f829cd85ca690b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:22.605272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-87564585-5efd-4cc7-913d-0b7b5b396781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:22.668780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1208/2401] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:22.680687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:22.682403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:22.704133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:22.739863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b54dcdc-fc96-48ea-b177-086effae39b9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:22.743456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:22.744073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:22.744350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:22.744623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:22.752826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:22.754243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:22.762972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1194/2402] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:22 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:25.504816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683151e9-d18b-413d-bbac-a8fbbc251184 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:25.505303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683151e9-d18b-413d-bbac-a8fbbc251184 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:25.505606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683151e9-d18b-413d-bbac-a8fbbc251184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:25.513648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:25.527722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-683151e9-d18b-413d-bbac-a8fbbc251184 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:25.528950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-683151e9-d18b-413d-bbac-a8fbbc251184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:25.584953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1209/2403] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:25:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:25.598936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:25.601611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:25.689286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6721e9a-135f-4c54-973e-b6c5e455f3eb service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:25.695472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:25.696281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:25.696659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:25.697028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:25.706254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:25.708111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e601b3b5-54b8-4f65-8437-dbbb6d0ca1ec service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:25.716937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1195/2404] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:25:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:25:26.282609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:26.283015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:26.283498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:26.320771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] MFA Rules not processed for user `b6d1952d79174b6596f829cd85ca690b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:26.322311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:26.332836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1210/2405] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:26 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:27.960345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bb2f9-3d6a-4a41-9cba-89213d39ca68 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:27.961473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bb2f9-3d6a-4a41-9cba-89213d39ca68 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:27.961839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5bb2f9-3d6a-4a41-9cba-89213d39ca68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:27.990918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6f5bb2f9-3d6a-4a41-9cba-89213d39ca68 None None] MFA Rules not processed for user `10a4469c46f64feb819f3b19a0f4e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:27.992748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6f5bb2f9-3d6a-4a41-9cba-89213d39ca68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:28.094782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1196/2406] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:27 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:31.123550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:31.125555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:31.262584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944b830d-0d45-4ed6-b230-8b9aa093aa3b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:31.277858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:31.277858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:31.278220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:31.278489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] PATH_INFO: `/v3/users/f0cc5dd7dbc142deb5f8813d149778ae` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:31.281466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0cc5dd7dbc142deb5f8813d149778ae)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:31.284189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:31.371903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0cc5dd7dbc142deb5f8813d149778ae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:31.388970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0cc5dd7dbc142deb5f8813d149778ae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:31.390480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0cc5dd7dbc142deb5f8813d149778ae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:31.403729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0cc5dd7dbc142deb5f8813d149778ae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:31.405380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0cc5dd7dbc142deb5f8813d149778ae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:31.455245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1211/2407] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:25:31 2022] DELETE /identity/v3/users/f0cc5dd7dbc142deb5f8813d149778ae => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:31.457535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:31.458028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:31.458793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:31.487612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:31.489094 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:31.552305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:31.565712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1197/2408] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:31.589209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:31.591246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:31.643365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:31.708378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40928d5-3909-4890-991e-539900afdc18 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:31.714416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:31.715324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:31.715979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:31.716604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:31.730006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:31.732072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9337bbab-d2b3-44d5-88e0-fa189c750871 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:31.744852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1212/2409] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:32.537582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:32.539076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:32.560337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bbe82-ad3c-4c59-996a-5e6a7a6d0f69 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:32.565509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:32.566421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:32.566849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:32.567258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] PATH_INFO: `/v3/projects/ba436ca4f3194b18840899392541fe2c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:32.569355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] RBAC: Authorizing `identity:delete_project(project_id=ba436ca4f3194b18840899392541fe2c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:32.571658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:32.697158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ba436ca4f3194b18840899392541fe2c', 'request_id': 'req-0f87b0da-9709-4b68-bab3-4f957e34443a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:32.714587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ba436ca4f3194b18840899392541fe2c', 'request_id': 'req-0f87b0da-9709-4b68-bab3-4f957e34443a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:32.737743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ba436ca4f3194b18840899392541fe2c', 'request_id': 'req-0f87b0da-9709-4b68-bab3-4f957e34443a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:32.790041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] The token cache is being invalidate because project ba436ca4f3194b18840899392541fe2c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:25:32.790396 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:32.792508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0f87b0da-9709-4b68-bab3-4f957e34443a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:32.797988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1198/2410] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:25:32 2022] DELETE /identity/v3/projects/ba436ca4f3194b18840899392541fe2c => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:32.799933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:32.800449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:32.800674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:32.852582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:32.853715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:33.067692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1213/2411] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:33.069071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.069357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.069633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.176925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:33.178800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:33.207535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1199/2412] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:33.212130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.226293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab157b0b-1f51-4307-961c-df9196e86520 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.239073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.241536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.242359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.243025 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.245017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:25:33.245017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.248425 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.286559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1214/2413] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:25:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:25:33.287782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.304921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37665808-5cd1-4455-8233-5a864fc6a599 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.315926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.316576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.316858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.317137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.317844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.319888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.393632 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1200/2414] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:25:33 2022] POST /identity/v3/projects => generated 389 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:25:33.398271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.413261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ae2abf-2011-4bbe-b98a-dd78ad137d3c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.556897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.557541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.557822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.558104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.561009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.570731 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.620897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1215/2415] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:25:33 2022] POST /identity/v3/users => generated 295 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:25:33.624957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.647895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82875998-4218-4778-83d5-96bde80b0947 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.661039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.661695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.662498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.662780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.663519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.665138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.685410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1201/2416] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:25:33.689864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.709431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaaa622-b69d-4682-82aa-61de3e540de3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.722864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.723540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.723819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.724092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.725330 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.726955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22d12fd-cbe3-4002-baec-f8a5c59b2e63 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.751855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1216/2417] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:25:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:25:33.756099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:33.771958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f2b832-b552-4f12-967c-6cd9f8120d54 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:33.783895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:33.784593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.784878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.785157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] PATH_INFO: `/v3/projects/08c3b68109db4aec9a808a216c12ad96/users/775c596375f844a5b77251f2584bdfb3/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.833529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=08c3b68109db4aec9a808a216c12ad96, user_id=775c596375f844a5b77251f2584bdfb3, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:33.836462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:33.890821 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1202/2418] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:25:33 2022] PUT /identity/v3/projects/08c3b68109db4aec9a808a216c12ad96/users/775c596375f844a5b77251f2584bdfb3/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:33.892668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860fa03-4840-46c1-ae1e-b4104fb1edf8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:33.893124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860fa03-4840-46c1-ae1e-b4104fb1edf8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:33.893229 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b860fa03-4840-46c1-ae1e-b4104fb1edf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:33.965209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b860fa03-4840-46c1-ae1e-b4104fb1edf8 None None] MFA Rules not processed for user `775c596375f844a5b77251f2584bdfb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:33.965415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b860fa03-4840-46c1-ae1e-b4104fb1edf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:34.005182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:34.107519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1217/2419] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:33 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:34.114505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:34.117403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:34.244435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc7af57-2924-467b-b694-da5c6c4e09bc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:34.349953 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:34.351033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:34.351764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:34.356716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:34.393357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:34.394899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a81553-9c74-4566-93ad-073c4cd880de service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:34.407814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1203/2420] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:34 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:44.097898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=19 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:44.401687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=81,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:52.273727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.274802 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.275178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.284647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1218/2421] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:25:52 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 31 14:25:52.292652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29023eb-35a1-49ae-8d15-7a70771c8bfc None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.293171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29023eb-35a1-49ae-8d15-7a70771c8bfc None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.293449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29023eb-35a1-49ae-8d15-7a70771c8bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.315503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a29023eb-35a1-49ae-8d15-7a70771c8bfc None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:52.317064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a29023eb-35a1-49ae-8d15-7a70771c8bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.412263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1204/2422] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:52.424252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:52.425955 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.483868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a880292f-d599-4575-b548-64f8d8ac0503 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:52.487288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:52.487930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.488210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.488488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.499526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:52.500991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae85bdd7-57cb-44de-8d0f-da7892fddec8 service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:52.511827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1219/2423] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:52.590436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.590814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.591162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] PATH_INFO: `/` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.592931 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1205/2424] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:25:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:25:52.598121 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2a286-9609-452f-977b-7b85c12a9b54 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.598410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2a286-9609-452f-977b-7b85c12a9b54 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.598688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce2a286-9609-452f-977b-7b85c12a9b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.619953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0ce2a286-9609-452f-977b-7b85c12a9b54 None None] MFA Rules not processed for user `a3c98dfd1f934a98b26889cda2d5c762`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:52.621180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0ce2a286-9609-452f-977b-7b85c12a9b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.673223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1220/2425] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:52.685558 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:52.687261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.700605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c28644-030a-4148-be74-87eaad6d2bc5 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:52.704142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:52.704747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.705029 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.705305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.712664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:52.714074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78eedc8-9fe5-4524-8982-ca1b90bc1bb7 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:52.721846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1206/2426] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:52.805710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.806002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.806274 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] PATH_INFO: `/` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.807664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1221/2427] 10.222.0.40 () {54 vars in 921 bytes} [Sun Jul 31 14:25:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 31 14:25:52.813049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.813334 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.813607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.832991 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] MFA Rules not processed for user `d04ab058ac79408f80cc485082c6591c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:52.834774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.844521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1207/2428] 10.222.0.40 () {58 vars in 1018 bytes} [Sun Jul 31 14:25:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:52.856066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:52.858071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:52.873888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256e8b56-276a-47db-8c45-7d7e5b9a0d81 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:52.877451 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:52.878067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:52.878401 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:52.878705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:52.889466 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:52.890906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ffa5f1-8f44-4656-896a-11b8497d1844 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:52.902234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1222/2429] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:25:53.367699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:53.434437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a04a1e-0a25-499f-ae5b-93ebc719bcb1 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:53.439832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:53.440679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:53.441019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:53.441302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] PATH_INFO: `/v3/users/775c596375f844a5b77251f2584bdfb3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:53.442593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=775c596375f844a5b77251f2584bdfb3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:53.444725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:53.506302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '775c596375f844a5b77251f2584bdfb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:53.523908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '775c596375f844a5b77251f2584bdfb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:53.524748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '775c596375f844a5b77251f2584bdfb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:53.525530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '775c596375f844a5b77251f2584bdfb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:53.532663 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '775c596375f844a5b77251f2584bdfb3'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:53.534730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:53.549559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1208/2430] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:25:53 2022] DELETE /identity/v3/users/775c596375f844a5b77251f2584bdfb3 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:53.554156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:53.555231 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:53.555655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:53.575704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:53.577030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:53.652156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1223/2431] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:53.662157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:53.663616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:53.722928 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dae0fc6-2c73-403c-b291-52a2dd98eb6a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:53.732119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:53.733035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:53.733428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:53.733811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:53.734623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:53.736142 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:53.744649 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:53.746081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:53.755923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1209/2432] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:25:53.781218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:25:53.814390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f10f26-51d8-4493-883d-adcadc63496a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:53.820820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:53.821931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:53.822370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:53.822483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:53.830236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:53.851867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:53.853790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc53e16e-bfae-4a9b-976b-19fce9c19f87 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:53.866459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1224/2433] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:25:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:25:54.079973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:54.081597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:54.096714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e332b29-27a4-4c0e-9bcc-ec925a5e205d service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:25:54.105697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:25:54.106552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:54.107043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:54.107171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] PATH_INFO: `/v3/projects/08c3b68109db4aec9a808a216c12ad96` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:54.108547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08c3b68109db4aec9a808a216c12ad96)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:25:54.110915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:25:54.185265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08c3b68109db4aec9a808a216c12ad96', 'request_id': 'req-b59dde8c-1b19-483d-82f0-956899365561'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:54.192336 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08c3b68109db4aec9a808a216c12ad96', 'request_id': 'req-b59dde8c-1b19-483d-82f0-956899365561'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:54.203589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08c3b68109db4aec9a808a216c12ad96', 'request_id': 'req-b59dde8c-1b19-483d-82f0-956899365561'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:54.218032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] The token cache is being invalidate because project 08c3b68109db4aec9a808a216c12ad96 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:25:54.218358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:54.218965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:25:54.221721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1210/2434] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:25:54 2022] DELETE /identity/v3/projects/08c3b68109db4aec9a808a216c12ad96 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:25:59.839437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4708-41bc-4371-a091-977a0acd8d98 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:25:59.839921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4708-41bc-4371-a091-977a0acd8d98 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:25:59.840125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0be4708-41bc-4371-a091-977a0acd8d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:25:59.858954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b0be4708-41bc-4371-a091-977a0acd8d98 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:25:59.860216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b0be4708-41bc-4371-a091-977a0acd8d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:25:59.924072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1225/2435] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:25:59 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:25:59.946954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:25:59.949082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:00.043071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b59dde8c-1b19-483d-82f0-956899365561 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:00.048035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:00.048675 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:00.049086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:00.049366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:00.058428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:00.060127 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:00.070057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1211/2436] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:25:59 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:00.493826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4061d-3879-4aec-a934-72b099027bb8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:00.494360 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4061d-3879-4aec-a934-72b099027bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:00.495022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd4061d-3879-4aec-a934-72b099027bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:00.523569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ffd4061d-3879-4aec-a934-72b099027bb8 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:00.525136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ffd4061d-3879-4aec-a934-72b099027bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:00.541050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1226/2437] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:26:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:00.558193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:00.575001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ede08e9-6b0f-4ebf-907a-cb2c749d696c service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:00.585550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:00.587349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:00.588686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:00.589146 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:00.597773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=90 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:00.602846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:00.604840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:00.618274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1212/2438] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:01.351889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4f18-5e3a-4507-876d-c7e46f5d0274 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:01.352400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4f18-5e3a-4507-876d-c7e46f5d0274 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:01.352489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ec4f18-5e3a-4507-876d-c7e46f5d0274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:01.375782 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-88ec4f18-5e3a-4507-876d-c7e46f5d0274 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:01.378079 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-88ec4f18-5e3a-4507-876d-c7e46f5d0274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:01.391538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1227/2439] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:26:01.410197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:01.421584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e29cec6c-8218-4bd2-b405-5207faa6e7c8 service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:01.426810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:01.427588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:01.427875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:01.428176 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:01.438739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:01.440756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:01.451479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1213/2440] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:01.734112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c3dd-7f85-4bf2-8f93-40b3d70759b8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:01.734418 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c3dd-7f85-4bf2-8f93-40b3d70759b8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:01.734911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c1c3dd-7f85-4bf2-8f93-40b3d70759b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:01.754176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-39c1c3dd-7f85-4bf2-8f93-40b3d70759b8 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:01.755346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-39c1c3dd-7f85-4bf2-8f93-40b3d70759b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:01.762681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1228/2441] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:26:01 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:01.775268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:01.786775 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c6326-2dc6-4a73-a487-935b7844989e service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:01.792745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:01.793577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:01.793861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:01.794164 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:01.804258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:01.805856 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:01.814300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1214/2442] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:02.174406 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:02.174692 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:02.174968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:02.195793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:02.197030 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:02.204708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1229/2443] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:26:02 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:02.225625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:02.240221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2b12af-9d4b-4883-b9b8-4ba2c72c29a0 service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:02.247064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:02.247064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:02.247064 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:02.247450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:02.257130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:02.258771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25601b5b-999a-4f40-9309-facec1ee9637 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:02.267951 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1215/2444] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:02 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:04.496409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:04.497849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:04.584806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81edc10d-bfe6-4ffe-93be-8582f10e7b4b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:04.589344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:04.589987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:04.590386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:04.590665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] PATH_INFO: `/v3/users/b6d1952d79174b6596f829cd85ca690b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:04.591983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6d1952d79174b6596f829cd85ca690b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:04.593902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:04.653041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6d1952d79174b6596f829cd85ca690b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:04.654910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:04.670934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6d1952d79174b6596f829cd85ca690b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:04.671661 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6d1952d79174b6596f829cd85ca690b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:04.679975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6d1952d79174b6596f829cd85ca690b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:04.680768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6d1952d79174b6596f829cd85ca690b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:04.699037 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1230/2445] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:26:04 2022] DELETE /identity/v3/users/b6d1952d79174b6596f829cd85ca690b => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:26:04.703926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:04.704447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:04.704562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:04.728716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:04.730208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:04.823605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1216/2446] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:04.831048 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:04.832536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:04.912312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-338ecb8b-46d6-4ba8-b3be-44266ec02667 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:04.916131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:04.916754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:04.917031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:04.917299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:04.938326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:04.939824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872fbf35-40d1-4ca0-b403-3e3dcc461cf2 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:04.952795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1231/2447] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:26:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:26:05.142939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.144437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:05.170476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a8dfb1-729d-4bb5-bc3b-c822f104a6a6 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.175249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.176025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.176303 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.177220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] PATH_INFO: `/v3/projects/b0ad86716ddf412f91a1128ba7159585` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.179197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0ad86716ddf412f91a1128ba7159585)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.181660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.246427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0ad86716ddf412f91a1128ba7159585', 'request_id': 'req-eebb127a-d8a0-4ec8-acd7-43a59b45671a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:05.256664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0ad86716ddf412f91a1128ba7159585', 'request_id': 'req-eebb127a-d8a0-4ec8-acd7-43a59b45671a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:05.264978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0ad86716ddf412f91a1128ba7159585', 'request_id': 'req-eebb127a-d8a0-4ec8-acd7-43a59b45671a'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:05.276022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] The token cache is being invalidate because project b0ad86716ddf412f91a1128ba7159585 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:26:05.276385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:05.277177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-eebb127a-d8a0-4ec8-acd7-43a59b45671a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:26:05.279132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1217/2448] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:26:05 2022] DELETE /identity/v3/projects/b0ad86716ddf412f91a1128ba7159585 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:26:05.284171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.284571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.284933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.312248 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:05.314318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:05.387440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1232/2449] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:05.395743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.396032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.396339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.420361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:05.421138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:05.432550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1218/2450] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:05.436794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.447307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84c789f-8145-48d5-bd33-99487561ca8b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.452609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.453991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.454390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.454765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.455854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:26:05.456262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.458510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.479965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1233/2451] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:26:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:26:05.481240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.494806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0186cb82-1b98-43e0-be12-d7aa07e7725a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.503036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.504049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.504393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.504942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.505683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.507813 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.541401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1219/2452] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:26:05 2022] POST /identity/v3/projects => generated 385 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:26:05.545449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.558315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c5139-366d-4ab7-b2fb-53299e4b26c3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.622043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:05.629144 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.629924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.630271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.630619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.631371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.633652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.657145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1234/2453] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:26:05 2022] POST /identity/v3/users => generated 293 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:26:05.661888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.675488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f156e48-bcb4-4684-8ac3-3c76767a46dd admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.680706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.681345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.681628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.681985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.682708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.684402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.687290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:05.702188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1220/2454] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:26:05.705896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.718592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c09d63d-a9fd-4052-ba5f-8cd9f88e1586 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.728747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.729620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.730040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.730387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.731588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.733960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466c07f2-20c8-489e-bfb1-8e31ab7639cb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.747801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1235/2455] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:26:05 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:26:05.751977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.762465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f5f859-30bb-498e-9b6c-69f24ffe44ba admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:05.767279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:05.767984 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.768265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.768540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] PATH_INFO: `/v3/projects/ea0d7081bf384d098fad20ef507f4dfd/users/3b003c297b914cb08f737b1f4d40b16d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.784509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea0d7081bf384d098fad20ef507f4dfd, user_id=3b003c297b914cb08f737b1f4d40b16d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:05.787796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:05.805697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1221/2456] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:26:05 2022] PUT /identity/v3/projects/ea0d7081bf384d098fad20ef507f4dfd/users/3b003c297b914cb08f737b1f4d40b16d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:26:05.810859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ed5e-99f9-4999-9fc3-b66baf26299b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:05.811151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ed5e-99f9-4999-9fc3-b66baf26299b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:05.811455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb4ed5e-99f9-4999-9fc3-b66baf26299b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:05.838067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-edb4ed5e-99f9-4999-9fc3-b66baf26299b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:05.839435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-edb4ed5e-99f9-4999-9fc3-b66baf26299b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:05.914455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1236/2457] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:05.924470 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:05.926216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:06.008920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-680020ce-e225-48a8-a5d2-fa23ad38a22a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:06.012873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:06.013510 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:06.013791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:06.014065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:06.026293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:06.028326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:06.043193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1222/2458] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:26:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:26:10.881730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:10.882252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:10.882539 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:10.912409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] MFA Rules not processed for user `3b003c297b914cb08f737b1f4d40b16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:10.913617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:11.006520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1237/2459] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:11.011249 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.027521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5140774-a4e8-4b8a-9551-7ffe12d5ac35 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.032384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.033034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.033309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.033579 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.034251 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.036926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.065107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1223/2460] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:26:11 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:26:11.069919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.081375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a38e3e-1700-485d-8e0f-98edfa0c2f90 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.143877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.144636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.145690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.146038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.146792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.149061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.166202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:11.181887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1238/2461] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:26:11 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:26:11.185954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.197580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4a2023-efc8-435b-a3fe-e5347d9944d6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.202966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.203556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.203836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.204109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.204814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.206425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.218773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1224/2462] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:26:11.223318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.233874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7124b6-095a-4225-95e0-881d341b4a48 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.239689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.240493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.240981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.241326 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] PATH_INFO: `/v3/projects/66b6d76f31f14cd2afd43cc377c35714/users/76a3927485aa4ebca964a71fdbb16a87/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.260268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b6d76f31f14cd2afd43cc377c35714, user_id=76a3927485aa4ebca964a71fdbb16a87, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.263290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.281545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1239/2463] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:26:11 2022] PUT /identity/v3/projects/66b6d76f31f14cd2afd43cc377c35714/users/76a3927485aa4ebca964a71fdbb16a87/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:26:11.285275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.297183 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb208d0-0ada-4e9c-ac69-43f51a4676de admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.364342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.365008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.365361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.365700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.366472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.368274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-302a567d-0ce2-4bab-ad1f-3fd7ce195c7d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.383320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1225/2464] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:26:11 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:26:11.383898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:11.394548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e22281-039b-4113-bb85-84e7a117bbd0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:11.400714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:11.401646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:11.402179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:11.402754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] PATH_INFO: `/v3/projects/66b6d76f31f14cd2afd43cc377c35714/users/76a3927485aa4ebca964a71fdbb16a87/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:11.411467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66b6d76f31f14cd2afd43cc377c35714, user_id=76a3927485aa4ebca964a71fdbb16a87, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:11.425931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:11.441300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1240/2465] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:26:11 2022] PUT /identity/v3/projects/66b6d76f31f14cd2afd43cc377c35714/users/76a3927485aa4ebca964a71fdbb16a87/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:26:15.942941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991f83-31e8-4ded-9274-d74b56266703 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:15.943399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991f83-31e8-4ded-9274-d74b56266703 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:15.943686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41991f83-31e8-4ded-9274-d74b56266703 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:15.973767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-41991f83-31e8-4ded-9274-d74b56266703 None None] MFA Rules not processed for user `76a3927485aa4ebca964a71fdbb16a87`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:15.974921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-41991f83-31e8-4ded-9274-d74b56266703 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:16.009909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:16.061353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1226/2466] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:15 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:16.076695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:16.082732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:16.196531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25625454-f2b9-4e6c-a231-e6fd7b3aa6d8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:16.201881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:16.203279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:16.203533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:16.203811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:16.277214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:16.278794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:16.295295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1241/2467] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:26:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:26:16.446511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70721-04b2-4dc8-89b7-e525c1d88aa2 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:16.446732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70721-04b2-4dc8-89b7-e525c1d88aa2 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:16.447007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace70721-04b2-4dc8-89b7-e525c1d88aa2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:16.467769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ace70721-04b2-4dc8-89b7-e525c1d88aa2 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:16.469290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ace70721-04b2-4dc8-89b7-e525c1d88aa2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:16.528168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1227/2468] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:26:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:16.540336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:16.541879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:16.613410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa4506c-61b0-4d98-80b7-f1bfa053b086 service glance] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:16.618322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:16.618936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:16.619212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:16.619520 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:16.627286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:16.628936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:16.637272 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1242/2469] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:22.161967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1b38-d795-41e6-9954-ecc7a70e164b None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:22.162920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1b38-d795-41e6-9954-ecc7a70e164b None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:22.163651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dc1b38-d795-41e6-9954-ecc7a70e164b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:22.216355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b1dc1b38-d795-41e6-9954-ecc7a70e164b None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:22.218196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b1dc1b38-d795-41e6-9954-ecc7a70e164b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:22.229688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1228/2470] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:26:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:22.251060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:22.270733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:22.284319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2290e47c-e72f-4c64-9440-2831877c669b service swift] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:22.303940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:22.304699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:22.305606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:22.306282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:22.324670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:22.327296 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76b349c-7abd-4f88-8d4f-9013e3f6df48 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:22.340010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1243/2471] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:27.392513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:27.393044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:27.393149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:27.414126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] MFA Rules not processed for user `3b003c297b914cb08f737b1f4d40b16d`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:27.415255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:27.422150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1229/2472] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:27 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:29.375844 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94918ab7-aad3-4a12-a7ce-0df02e076c9f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:29.376429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94918ab7-aad3-4a12-a7ce-0df02e076c9f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:29.376544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94918ab7-aad3-4a12-a7ce-0df02e076c9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:29.396627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-94918ab7-aad3-4a12-a7ce-0df02e076c9f None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:29.397970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-94918ab7-aad3-4a12-a7ce-0df02e076c9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:29.405699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1244/2473] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:26:29 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:29.420546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:29.433347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717ece9e-ab9c-4e59-8e2e-756345512647 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:29.438400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:29.439178 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:29.439575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:29.439922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:29.449023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:29.450884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:29.460640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1230/2474] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:26:29 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:26:39.390716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=8 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:39.456364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=53 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:41.717616 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf648-f972-4b23-8076-2449ae226a72 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:41.718114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf648-f972-4b23-8076-2449ae226a72 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:41.718318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b03bf648-f972-4b23-8076-2449ae226a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:41.736883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b03bf648-f972-4b23-8076-2449ae226a72 None None] MFA Rules not processed for user `d6e8399080a942f1b689fc8c6b42c717`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:26:41.738138 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b03bf648-f972-4b23-8076-2449ae226a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:26:41.790385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1245/2475] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:26:41 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:26:51.783198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=25 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:26:56.270870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:26:56.335221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91ad5ec-9122-4efe-8384-b515f97c868f service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:26:56.339796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:26:56.340611 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:26:56.341071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:26:56.341356 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:26:56.352618 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:26:56.354315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252a5943-ba91-4786-86a2-63eff20ff114 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:26:56.370914 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1231/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:26:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:27:06.364987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=26 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:27:10.341636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:10.341636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:10.341636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:10.360956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] MFA Rules not processed for user `10a4469c46f64feb819f3b19a0f4e89b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:27:10.362734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:10.420873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1246/2477] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:27:10 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:27:12.714136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:12.714653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:12.714759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:12.732963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] MFA Rules not processed for user `53dab2ec3f2e4973893fb838c3d1d545`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:27:12.734152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:12.788576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1232/2478] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:27:12 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:27:20.414328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=25 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:27:22.781823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=25 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:27:38.690483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:27:38.700082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0342c62d-ddcf-4220-94d9-ef12320f0b66 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:27:38.703953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:27:38.704591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:38.704874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:38.705153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:38.707438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:38.775290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:27:38.776742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:38.787287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:27:38.788665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516c5b23-e964-4d82-9c72-b036ab14cfa0 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:27:38.788797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:27:38.792576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:27:38.793954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:38.794354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:38.794633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:38.796421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:38.800234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1247/2479] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:27:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:27:38.810429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:27:38.811901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:27:38.824159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1233/2480] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:27:38 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:27:47.380496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:27:47.381913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:47.441545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e074d-3a0e-4643-9f28-6d93aedf8d9a service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:27:47.445980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:27:47.446601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:47.446880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:47.447154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:47.449132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:47.460883 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:27:47.462327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27fcf58-6514-43b7-aebd-066549fb87ad service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:27:47.474840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1248/2481] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:27:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:27:48.819651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=6 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:27:57.467567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=60 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:27:59.576195 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:27:59.578162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:59.593175 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50ba99d-e34c-41e2-b39c-060e9b214fb6 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:27:59.599355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:27:59.600002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:27:59.600282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:27:59.600553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:27:59.602327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:27:59.613228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:27:59.614831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:27:59.624381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1234/2482] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:27:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:01.667527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:01.668059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:01.668167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:01.687067 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] MFA Rules not processed for user `d6e8399080a942f1b689fc8c6b42c717`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:01.688310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:01.695156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1249/2483] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:01 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:02.048000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:02.061415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f705f39e-4646-4287-b692-8ae2a22f501d service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:02.068706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:02.070560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:02.071325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:02.071325 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:02.076236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:02.101581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:02.104393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:02.118388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1235/2484] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:02.467251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:02.468913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:02.554746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76842f4e-118a-41a0-9f9a-676047ebcb71 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:02.559897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:02.560528 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:02.560818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:02.561090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:02.562847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:02.575548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:02.577178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:02.587089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1250/2485] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:06.242749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:06.252112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfb33d26-8367-4186-a63e-68241ee93d20 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:06.256665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:06.257277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:06.257553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:06.257819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:06.259704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:06.269973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:06.271486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:06.279497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1236/2486] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:11.424687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:11.438424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be1bb46-54d1-43d8-bad5-cfc7c7f4463d service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:11.444578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:11.445189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:11.445468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:11.445744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:11.447523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:11.458214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:11.459811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:11.468708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1251/2487] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:16.276016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=18 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:16.883027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:16.895817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab9e37a-8be6-4624-98f8-e0499b361c45 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:16.901256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:16.901859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:16.902148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:16.902402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:16.904131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:16.915825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:16.917380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:16.928047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1237/2488] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:18.755114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:18.764915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a0a12c-23f9-4d2e-84d4-7cfb35980bdd service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:18.768570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:18.769188 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:18.769468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:18.769768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:18.771572 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:18.844530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:18.845982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:18.856717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1252/2489] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:22.679351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:22.680989 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:22.696107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3054d0c7-f793-42c5-9044-e16ac0c41646 service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:22.700998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:22.701666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:22.701941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:22.702211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] PATH_INFO: `/v3/users/10a4469c46f64feb819f3b19a0f4e89b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:22.703404 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] RBAC: Authorizing `identity:delete_user(user_id=10a4469c46f64feb819f3b19a0f4e89b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:22.705342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:22.752002 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10a4469c46f64feb819f3b19a0f4e89b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:22.763870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10a4469c46f64feb819f3b19a0f4e89b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:22.764651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10a4469c46f64feb819f3b19a0f4e89b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:22.765253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10a4469c46f64feb819f3b19a0f4e89b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:22.772714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4806e39a-4f92-48dc-a5bc-2360afc9261d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10a4469c46f64feb819f3b19a0f4e89b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:22.787013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1238/2490] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:22 2022] DELETE /identity/v3/users/10a4469c46f64feb819f3b19a0f4e89b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:22.979645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:22.993332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ba60109-9a7f-406b-89c6-c5f5c54966ba service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.056919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.057561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.057842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.058124 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] PATH_INFO: `/v3/projects/832629dd8594494ab41b6596b2d73076` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.059265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] RBAC: Authorizing `identity:delete_project(project_id=832629dd8594494ab41b6596b2d73076)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.061200 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.067904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:23.107135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '832629dd8594494ab41b6596b2d73076', 'request_id': 'req-faa9b89e-0459-4f13-87ca-01f9f5cb4286'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:23.113916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '832629dd8594494ab41b6596b2d73076', 'request_id': 'req-faa9b89e-0459-4f13-87ca-01f9f5cb4286'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:23.119957 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '832629dd8594494ab41b6596b2d73076', 'request_id': 'req-faa9b89e-0459-4f13-87ca-01f9f5cb4286'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:23.134745 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] The token cache is being invalidate because project 832629dd8594494ab41b6596b2d73076 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:23.135034 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:23.135726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-faa9b89e-0459-4f13-87ca-01f9f5cb4286 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:23.137244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1253/2491] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:22 2022] DELETE /identity/v3/projects/832629dd8594494ab41b6596b2d73076 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:23.142144 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.142451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.142732 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.163003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:23.164296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:23.225679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1239/2492] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:23.233489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.233777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.234051 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.254062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:23.255282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:23.263460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1254/2493] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:23.267744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.277921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410c81bb-4692-47fa-a0cd-88d8c791a851 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.282533 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.283148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.283438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.283739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.284441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:28:23.284777 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.286366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.297833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1240/2494] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:28:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:28:23.302095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.311541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4d29e9-3b6e-495d-a73a-295073d1d13b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.316746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.317352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.317632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.317906 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.318564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.320503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.348647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1255/2495] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:23.352825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.363755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034e7e84-a2a8-43bb-86ba-9f9c579dc607 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.405248 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:23.418383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.418990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.419262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.419551 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.420214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.422108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.446684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1241/2496] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/users => generated 278 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:28:23.447678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.458910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57d6ab1-af06-4ed5-aa58-88c889c12520 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.463670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.464375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.464724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.465066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.465773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.467357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.477363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1256/2497] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:28:23.481299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.491621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e99139-2b8e-4087-a56f-93c3ecfefae5 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.496132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.496730 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.497005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.497275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.497966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.499574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b7e5ef-d04d-450e-95d2-1777500cca21 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.510469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1242/2498] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:23.514376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.524456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7279ecc3-e3e6-4c9a-a9b4-183645bd6220 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.529166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.529822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.530104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.530379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] PATH_INFO: `/v3/projects/d201825147894f54bb378045f8a50acd/users/9b8382eb860140389b45a151a61d1ad8/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.544721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d201825147894f54bb378045f8a50acd, user_id=9b8382eb860140389b45a151a61d1ad8, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.547355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.560869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1257/2499] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:23 2022] PUT /identity/v3/projects/d201825147894f54bb378045f8a50acd/users/9b8382eb860140389b45a151a61d1ad8/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:23.573036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.573439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.573724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.600830 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] MFA Rules not processed for user `9b8382eb860140389b45a151a61d1ad8`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:23.602034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:23.663817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1243/2500] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:23.668397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.733373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ee0718-a191-430a-8ffd-318d35bed934 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.738669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.739310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.739627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.739905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.740570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.742471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.769061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1258/2501] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/projects => generated 357 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:28:23.773095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.783589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fd065-a6aa-4e07-87ac-2beaab4488c3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.853254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.853884 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.854157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.854448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.855285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.857313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.878155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1244/2502] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:23 2022] POST /identity/v3/users => generated 285 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:23.882252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.892450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42993454-a214-4da4-8a74-01bf9feaaf6b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.896963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.897587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.897868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.898135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.898836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.900435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.913924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1259/2503] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:23.916370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.927549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed6a70-2105-4987-b9d3-b3426222c832 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:23.935627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:23.936515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:23.936833 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:23.937111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] PATH_INFO: `/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34/users/25df802fddd5483fa1d23d03a08c7358/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:23.954095 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e1b2a3e0174c1ea07a944da177ef34, user_id=25df802fddd5483fa1d23d03a08c7358, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:23.956943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:23.970008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1245/2504] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:23 2022] PUT /identity/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34/users/25df802fddd5483fa1d23d03a08c7358/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:23.974492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:23.984279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:23.990002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83358a10-2b2d-435a-b91c-539d4e3ff409 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:24.053653 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:24.054281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:24.054564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:24.054841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:24.055601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:24.057834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cef4441-e9e2-4548-a221-863a21505e17 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:24.067450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1260/2505] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:23 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:24.071685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:24.082972 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6980efa2-969f-4836-a976-9f0ef27bde8d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:24.089245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:24.090369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:24.090817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:24.091102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] PATH_INFO: `/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34/users/25df802fddd5483fa1d23d03a08c7358/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:24.093128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d3e1b2a3e0174c1ea07a944da177ef34, user_id=25df802fddd5483fa1d23d03a08c7358, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:24.095827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:24.107478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1246/2506] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:24 2022] PUT /identity/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34/users/25df802fddd5483fa1d23d03a08c7358/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:28:24.117717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:24.118089 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:24.118491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:24.149432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] MFA Rules not processed for user `25df802fddd5483fa1d23d03a08c7358`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:24.150683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:24.224903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1261/2507] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:24 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:24.236217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:24.237924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:24.247546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:24.296532 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f66d7e-00e6-4056-a30d-8a6307cbdb03 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:24.300219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:24.300831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:24.301103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:24.301369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:24.310720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:24.312170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97931cdf-792e-4a61-920e-cbc0b869ee2b service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:24.321413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1247/2508] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:24 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:24.398305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:24.408132 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1e7f56-7100-4023-bd5c-36eed6e1c5b2 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:24.412336 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:24.412941 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:24.413220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:24.413495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:24.476068 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:24.477412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:24.486300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1262/2509] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:24 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:33.476254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:33.476733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:33.477257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:33.496072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:33.497547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:33.558742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1248/2510] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:28:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:33.609771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:33.612500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:33.672609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:33.733053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b182f22-b930-484b-bd3c-04abe43bb2fc service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:33.739728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:33.740282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:33.740657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:33.740815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:33.753689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:33.755683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:33.771446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1263/2511] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:28:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:28:35.246771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:35.248758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:35.379234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7bf279-3e82-4c8e-9cef-19658c9cca1d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:35.388028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:35.388915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:35.389351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:35.389781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:35.391867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:35.405650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:35.407314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:35.425878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1249/2512] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:28:36.610366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:36.637020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debfc659-4317-4757-95a7-feab0dfdb9e5 service swift] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:36.657306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:36.660136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:36.661604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:36.662038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:36.666594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:36.715618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:36.718192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a748c07-7858-4de9-9af2-a3a5c43f0984 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:36.749151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1264/2513] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:40.174658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:40.228158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:40.251528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3027b5-adc2-4370-a227-62b4aaa748d3 service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:40.257797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:40.258511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:40.258872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:40.259209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] PATH_INFO: `/v3/users/9b8382eb860140389b45a151a61d1ad8` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:40.260621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9b8382eb860140389b45a151a61d1ad8)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:40.262743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:40.327491 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9b8382eb860140389b45a151a61d1ad8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:40.346944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9b8382eb860140389b45a151a61d1ad8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:40.347981 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9b8382eb860140389b45a151a61d1ad8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:40.348595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9b8382eb860140389b45a151a61d1ad8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:40.361188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9b8382eb860140389b45a151a61d1ad8'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:40.387882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1250/2514] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:40 2022] DELETE /identity/v3/users/9b8382eb860140389b45a151a61d1ad8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:40.392362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:40.393011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:40.393011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:40.415757 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:40.437792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:40.491462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1265/2515] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:40.499544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:40.501031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:40.576301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d30f22-cb4a-4272-96b5-d08eccae8061 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:40.601101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:40.601757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:40.602037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:40.602311 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:40.612811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:40.614263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:40.625328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1251/2516] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:41.100374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:41.105555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:41.134431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00951c2-d597-4563-9904-b8e3500d6393 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:41.144585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:41.144585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:41.144585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:41.144585 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] PATH_INFO: `/v3/projects/d201825147894f54bb378045f8a50acd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:41.145020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d201825147894f54bb378045f8a50acd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:41.147956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:41.208590 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd201825147894f54bb378045f8a50acd', 'request_id': 'req-a1d4d060-998c-4f98-8a59-9260301ea052'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.216519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd201825147894f54bb378045f8a50acd', 'request_id': 'req-a1d4d060-998c-4f98-8a59-9260301ea052'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.224242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd201825147894f54bb378045f8a50acd', 'request_id': 'req-a1d4d060-998c-4f98-8a59-9260301ea052'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.239459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] The token cache is being invalidate because project d201825147894f54bb378045f8a50acd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:41.239998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.240836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.242956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1266/2517] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:41 2022] DELETE /identity/v3/projects/d201825147894f54bb378045f8a50acd => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:41.247318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:41.258272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4dec6b8-5041-4e9e-ad99-ce3b200eba9a service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:41.337785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:41.338484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:41.338838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:41.339185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] PATH_INFO: `/v3/users/25df802fddd5483fa1d23d03a08c7358` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:41.340752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25df802fddd5483fa1d23d03a08c7358)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:41.342939 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:41.346993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:41.468395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25df802fddd5483fa1d23d03a08c7358'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.490497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25df802fddd5483fa1d23d03a08c7358'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.491353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25df802fddd5483fa1d23d03a08c7358'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.492331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25df802fddd5483fa1d23d03a08c7358'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.501368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a70958fe-b731-4e71-9348-f18666f7e8e5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25df802fddd5483fa1d23d03a08c7358'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:41.537014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1252/2518] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:41 2022] DELETE /identity/v3/users/25df802fddd5483fa1d23d03a08c7358 => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:41.909072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:41.910821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:41.937529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d4d060-998c-4f98-8a59-9260301ea052 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:41.965046 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:42.043352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.044335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.044780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.045222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] PATH_INFO: `/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.046589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d3e1b2a3e0174c1ea07a944da177ef34)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.048743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.107821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd3e1b2a3e0174c1ea07a944da177ef34', 'request_id': 'req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:42.108138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.108429 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.108705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.120368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd3e1b2a3e0174c1ea07a944da177ef34', 'request_id': 'req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:42.135208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd3e1b2a3e0174c1ea07a944da177ef34', 'request_id': 'req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:42.137933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:42.139451 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:42.159648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] The token cache is being invalidate because project d3e1b2a3e0174c1ea07a944da177ef34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:42.160018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:42.160624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-f087a70a-bbc3-4e4c-bf89-3c6ee4b9fdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:42.162369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1267/2519] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:41 2022] DELETE /identity/v3/projects/d3e1b2a3e0174c1ea07a944da177ef34 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:42.168537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8b81-6133-4b0b-b976-7165a0450f6a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.169378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8b81-6133-4b0b-b976-7165a0450f6a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.169886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-365a8b81-6133-4b0b-b976-7165a0450f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.197224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-365a8b81-6133-4b0b-b976-7165a0450f6a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:42.198823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-365a8b81-6133-4b0b-b976-7165a0450f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:42.222863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1253/2520] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:42.243877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.245320 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:42.300493 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1268/2521] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:42.302136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.302433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.302717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.331642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:42.333013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:42.341271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f36dca-567f-436a-a81a-69228dc4464d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.345157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1269/2522] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:42.346151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.347030 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.347765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.348118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.348569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.356764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20004f9-d0a5-4b4b-b08a-5bff72b8c866 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.359958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.362150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.362291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.363090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.363462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.363817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.364588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:28:42.364965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.366623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.375882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1254/2523] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:28:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 14:28:42.380301 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1270/2524] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:28:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:28:42.388456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.400462 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc18032-2a5e-4426-a86a-44256fbb9dd8 service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.405476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.406135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.406458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.406814 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.407578 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.409713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.429461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:42.454447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1255/2525] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/projects => generated 387 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:42.458736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.475196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7220066-c7f7-47be-9586-b429ea9e35cb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.522316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:42.559681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.560322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.560603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.561105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.561678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.563703 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.597084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1271/2526] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/users => generated 294 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:42.601373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.618816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d511c38-cd62-4676-af6e-320c87a06664 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.635358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.636244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.636870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.637174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.638155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.644060 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.661556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1256/2527] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:42.665679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.682097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae14e761-b15d-4ebb-9f79-b9db83b262cf admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.687893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.688736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.689271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.690542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.691515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.693487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b58bdf-8c75-442f-8d6c-1291fa7bafdb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.718621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1272/2528] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:42.719886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:42.739772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36b22-b83e-4cfd-a884-8ec72088e138 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:42.745366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:42.746159 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.746508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.746861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] PATH_INFO: `/v3/projects/7450181f0c064d83b4e09eba95ce718d/users/3456f0b1665a45fa915c64ae42c8dc80/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.768935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=7450181f0c064d83b4e09eba95ce718d, user_id=3456f0b1665a45fa915c64ae42c8dc80, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:42.771861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:42.797665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1257/2529] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:42 2022] PUT /identity/v3/projects/7450181f0c064d83b4e09eba95ce718d/users/3456f0b1665a45fa915c64ae42c8dc80/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:42.812013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:42.812487 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:42.812803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:42.871039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] MFA Rules not processed for user `3456f0b1665a45fa915c64ae42c8dc80`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:42.872614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:42.963241 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1273/2530] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:42.967371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.076314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04321a4-0e9f-4477-9fc9-7a35a8b947cf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.076314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.076314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.076314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.076797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.076797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.076797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.128321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1258/2531] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:42 2022] POST /identity/v3/projects => generated 387 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:28:43.132266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.153011 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50083ed2-e400-4cff-8f95-fb009b25f097 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.238610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.239318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.239672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.239980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.240672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.242734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.273052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1274/2532] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:43 2022] POST /identity/v3/users => generated 300 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:43.276377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.289432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f898df4e-3ec9-413d-8566-b50d7dbb2f46 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.294605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.297143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.297143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.297143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.297553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.299923 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.312113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1259/2533] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:43.316388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.333268 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1215d32b-6c93-4959-9bee-128241bfa86f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.341669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.342365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.342652 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.342932 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] PATH_INFO: `/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52/users/74c90225efd54b6bba9735bb3bc15b10/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.361422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94ef29a8b674a5dbb82841d4ef67f52, user_id=74c90225efd54b6bba9735bb3bc15b10, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.364922 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.385862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1275/2534] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:43 2022] PUT /identity/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52/users/74c90225efd54b6bba9735bb3bc15b10/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:43.389556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.400869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49d1d53-b8c1-45aa-8bba-27126e95dd49 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.495503 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.497000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.497452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.497890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.498763 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.500671 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ddd139b-ced3-48e6-b6d7-379d152c16aa admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.519904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1260/2535] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:43 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:43.521509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.533600 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206f9a75-58a4-4c6b-a119-beb40fef1717 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.540726 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.541716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.542167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.542609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] PATH_INFO: `/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52/users/74c90225efd54b6bba9735bb3bc15b10/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.544789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d94ef29a8b674a5dbb82841d4ef67f52, user_id=74c90225efd54b6bba9735bb3bc15b10, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.550503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.554204 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:43.581264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1276/2536] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:43 2022] PUT /identity/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52/users/74c90225efd54b6bba9735bb3bc15b10/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:43.583005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.583623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.584072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.622060 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] MFA Rules not processed for user `74c90225efd54b6bba9735bb3bc15b10`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:43.623461 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:43.647565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:43.715857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1261/2537] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:43 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:43.728914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:43.737073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:43.829840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc1b819-7075-44c3-a51b-7408f751eb9b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:43.834577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:43.835357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:43.835774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:43.836060 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:43.849422 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:43.852052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697cdd28-c9f2-4c94-89cd-b757ecdab828 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:43.872664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1277/2538] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:43 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:44.024118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:44.193534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b448c01-6c8e-4669-95b3-454a887091ed None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:44.209044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:44.209918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:44.210305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:44.210685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] PATH_INFO: `/v3/users/3456f0b1665a45fa915c64ae42c8dc80` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:44.212044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3456f0b1665a45fa915c64ae42c8dc80)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:44.214260 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:44.294221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3456f0b1665a45fa915c64ae42c8dc80'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:44.310481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3456f0b1665a45fa915c64ae42c8dc80'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:44.311255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3456f0b1665a45fa915c64ae42c8dc80'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:44.311955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3456f0b1665a45fa915c64ae42c8dc80'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:44.329188 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3456f0b1665a45fa915c64ae42c8dc80'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:44.358314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1262/2539] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:44 2022] DELETE /identity/v3/users/3456f0b1665a45fa915c64ae42c8dc80 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:44.363041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:44.363579 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:44.364383 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:44.392513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:44.394150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:44.476441 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1278/2540] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:44.490141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:44.491817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:44.547782 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:44.600414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255297-afa8-48a6-a5ff-0929a6dab422 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:44.605439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:44.608988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:44.608988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:44.608988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:44.621372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:44.622824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:44.636289 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1263/2541] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:45.008221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:45.009689 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:45.022619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd219fe-bfaf-4083-a844-36e478cfb234 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:45.027905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:45.028574 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:45.029001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:45.029294 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] PATH_INFO: `/v3/projects/7450181f0c064d83b4e09eba95ce718d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:45.031002 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7450181f0c064d83b4e09eba95ce718d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:45.034111 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:45.094879 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7450181f0c064d83b4e09eba95ce718d', 'request_id': 'req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.105257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7450181f0c064d83b4e09eba95ce718d', 'request_id': 'req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.113340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7450181f0c064d83b4e09eba95ce718d', 'request_id': 'req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.129199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] The token cache is being invalidate because project 7450181f0c064d83b4e09eba95ce718d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:45.129630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.130429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.132070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1279/2542] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:45 2022] DELETE /identity/v3/projects/7450181f0c064d83b4e09eba95ce718d => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:45.136131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:45.146308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa11b712-4c43-4f3f-b812-abe73d070e81 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:45.225720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:45.226492 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:45.227215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:45.227215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] PATH_INFO: `/v3/users/74c90225efd54b6bba9735bb3bc15b10` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:45.229433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c90225efd54b6bba9735bb3bc15b10)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:45.232873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:45.295626 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c90225efd54b6bba9735bb3bc15b10'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.310687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c90225efd54b6bba9735bb3bc15b10'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.312307 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c90225efd54b6bba9735bb3bc15b10'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.313024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c90225efd54b6bba9735bb3bc15b10'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.324665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7ecb3a19-9d80-4e89-a7e9-b247e03458c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c90225efd54b6bba9735bb3bc15b10'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:45.345697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1264/2543] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:45 2022] DELETE /identity/v3/users/74c90225efd54b6bba9735bb3bc15b10 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:45.825940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:45.828115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:45.850218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a02edfb-ca50-40dd-b3c1-cfebd81c5ba3 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:45.872114 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:45.932903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:45.933622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:45.933835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:45.934109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] PATH_INFO: `/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:45.935258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] RBAC: Authorizing `identity:delete_project(project_id=d94ef29a8b674a5dbb82841d4ef67f52)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:45.937221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:45.999375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd94ef29a8b674a5dbb82841d4ef67f52', 'request_id': 'req-87a17d00-f512-4ca1-bf37-10607e66521b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:46.010024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd94ef29a8b674a5dbb82841d4ef67f52', 'request_id': 'req-87a17d00-f512-4ca1-bf37-10607e66521b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:46.019707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd94ef29a8b674a5dbb82841d4ef67f52', 'request_id': 'req-87a17d00-f512-4ca1-bf37-10607e66521b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:46.035885 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] The token cache is being invalidate because project d94ef29a8b674a5dbb82841d4ef67f52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:46.036749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:46.037728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-87a17d00-f512-4ca1-bf37-10607e66521b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:46.039989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1280/2544] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:45 2022] DELETE /identity/v3/projects/d94ef29a8b674a5dbb82841d4ef67f52 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:46.044872 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.045728 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.046209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.072982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:46.074616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:46.166890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:46.202107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1265/2545] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:46.208295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.208589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.208964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.242424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:46.244888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:46.269796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1281/2546] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:46.270700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.284818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db6c90b-0a3c-49c6-862c-78c6ff96fb36 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.295443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.296887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.297610 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.298076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.299033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:28:46.299575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.301367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.314291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1266/2547] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:28:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:28:46.319781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.337461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ba704c-38d1-4423-9dc2-3c69efc86076 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.344634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.345760 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.346313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.347003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.348257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.376983 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.424064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1282/2548] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:46 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:28:46.428528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.439174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a342c0ad-5543-471e-acc9-def29075ed83 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.506697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.507344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.507818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.508101 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.508780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.511085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.544630 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1267/2549] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:46 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:46.545788 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.556811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1ba69e-8cb0-4b7a-adcd-c2aabc937812 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.561506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.562127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.562411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.562685 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.563379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.565564 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.576225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1283/2550] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:28:46.580134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.595354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d48d46-eb58-4f81-b59f-f8bcaec85a30 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.600529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.601155 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.601439 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.601719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.602431 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.604209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b40c918-4dbd-4cae-a635-96048f9d4664 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.616244 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1268/2551] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:46.620701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.633624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fef974-7154-4bf1-a03d-c16a7eb8b6fb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.641289 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.642442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.642442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.642977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] PATH_INFO: `/v3/projects/784093cb66924ab693e909de209d6dc0/users/55fc4b9052b54389991cd528093564c6/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.663394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=784093cb66924ab693e909de209d6dc0, user_id=55fc4b9052b54389991cd528093564c6, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.666660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.687481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1284/2552] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:46 2022] PUT /identity/v3/projects/784093cb66924ab693e909de209d6dc0/users/55fc4b9052b54389991cd528093564c6/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:46.695628 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.695924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.696191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.731954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] MFA Rules not processed for user `55fc4b9052b54389991cd528093564c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:46.733304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:46.850317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1269/2553] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:46.865603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:46.865603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:46.928535 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:46.932623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2274fb-73d2-4351-ad20-8dc5804f3f73 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:46.938149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:46.938958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:46.939321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:46.940636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:46.952817 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:46.954369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f198e6-cca5-43ee-a825-15bbf90a97fb service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:46.964934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1285/2554] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:46.989846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:47.047134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:47.061218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3afff02-58ed-4a95-9717-7d26d2646713 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:47.067694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:47.068472 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:47.068859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:47.069238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] PATH_INFO: `/v3/users/55fc4b9052b54389991cd528093564c6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:47.070647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fc4b9052b54389991cd528093564c6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:47.072696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:47.148714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fc4b9052b54389991cd528093564c6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.171614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fc4b9052b54389991cd528093564c6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.177902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fc4b9052b54389991cd528093564c6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.178574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fc4b9052b54389991cd528093564c6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.186641 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fc4b9052b54389991cd528093564c6'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.203376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1270/2555] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:46 2022] DELETE /identity/v3/users/55fc4b9052b54389991cd528093564c6 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:47.208143 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:47.208434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:47.208717 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:47.228271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:47.229279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:47.251331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:47.254052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:47.305826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1286/2556] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:47.321496 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:47.323513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:47.341891 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7d7a56-b71e-4506-aceb-9f42fe35dd32 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:47.347883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:47.348653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:47.349028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:47.349408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:47.351327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:47.363864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:47.365933 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:47.378452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1271/2557] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:47.391681 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa33918f-9a33-4078-9a89-86839e610076 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:47.395956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:47.396648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:47.396991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:47.397350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:47.407440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:47.409603 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193b499-ee2e-47ea-9be6-9619c0beaf61 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:47.420242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1287/2558] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:47.878982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:47.880507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:47.895850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66269c58-8806-480a-8b78-78b649501a1f service placement] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:47.901560 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:47.902715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:47.903197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:47.903570 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] PATH_INFO: `/v3/projects/784093cb66924ab693e909de209d6dc0` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:47.905087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] RBAC: Authorizing `identity:delete_project(project_id=784093cb66924ab693e909de209d6dc0)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:47.908576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:47.979564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '784093cb66924ab693e909de209d6dc0', 'request_id': 'req-98cd52ef-c504-41fb-b4f6-accfaea3185e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.990088 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '784093cb66924ab693e909de209d6dc0', 'request_id': 'req-98cd52ef-c504-41fb-b4f6-accfaea3185e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:47.992083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:48.003033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '784093cb66924ab693e909de209d6dc0', 'request_id': 'req-98cd52ef-c504-41fb-b4f6-accfaea3185e'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:48.032999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] The token cache is being invalidate because project 784093cb66924ab693e909de209d6dc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:48.033924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:48.034600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-98cd52ef-c504-41fb-b4f6-accfaea3185e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:48.039909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1272/2559] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:47 2022] DELETE /identity/v3/projects/784093cb66924ab693e909de209d6dc0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:48.041596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.054081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.054081 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.071978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:48.073254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:48.137287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1288/2560] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:48.146160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.146698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.146807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.169232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:48.170469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:48.180805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1273/2561] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:48.184950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.195453 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc0d16-882d-4c0d-8036-f6e06dd87bc9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.200599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.201222 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.201500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.201768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.202618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:28:48.202741 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.204773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.219463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1289/2562] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:28:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:28:48.222857 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.235402 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32fa9094-26a0-4ea5-b937-58e8f613aadd None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.241563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.242118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.242394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.242661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.243901 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.245332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.279557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1274/2563] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:48 2022] POST /identity/v3/projects => generated 373 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:48.283776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.294215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22e4ac-50a2-4c9e-8711-9b975762a450 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.316357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:48.355822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.356443 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.356720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.356991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.357647 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.359578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.381935 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1290/2564] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:48 2022] POST /identity/v3/users => generated 287 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:48.386217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.397022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02d3d21-57f0-4920-865b-f8151e5aa715 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.401843 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.402458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.402724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.402996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.403742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.405331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.415930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1275/2565] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:48.422224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.434438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91328cd-a51a-40e0-b137-47732db0b873 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.440432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.441281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.441627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.441974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.442750 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.445380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd007c22-5a29-4002-ab30-cd3d67f4547b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.458267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1291/2566] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:48 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:48.472950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.492147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1784b3-35e3-4609-ab2e-a6e28c612ac3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.498802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.499613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.499889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.500160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] PATH_INFO: `/v3/projects/857f68541e904e7abd38151d6c72d687/users/edfb7c31479e4cc6bd228af8e93b3dfa/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.521292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=857f68541e904e7abd38151d6c72d687, user_id=edfb7c31479e4cc6bd228af8e93b3dfa, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.524215 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.543673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1276/2567] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:48 2022] PUT /identity/v3/projects/857f68541e904e7abd38151d6c72d687/users/edfb7c31479e4cc6bd228af8e93b3dfa/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:48.555589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.556799 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.557281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.605399 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] MFA Rules not processed for user `edfb7c31479e4cc6bd228af8e93b3dfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:48.607309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:48.730543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1292/2568] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:48 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:48.742383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:48.744442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:48.802130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdb2ff7-0cd2-46cd-990c-416c67d34db3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:48.805704 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:48.806347 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:48.806620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:48.806888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:48.815237 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:48.816695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:48.825890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1277/2569] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:50.956438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:50.958378 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:51.044376 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:51.046267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:51.092485 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f42fce-e2ac-4211-819b-c7ea71d4c3f5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:51.098213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:51.098928 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:51.100122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:51.100122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:51.101759 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:51.116387 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:51.118201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:51.131290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1293/2570] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:51.147043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5ab1af-38d4-4f66-ac17-ef0a59bf90d1 service cinder] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:51.152111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:51.152755 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:51.153018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:51.153457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] PATH_INFO: `/v3/users/3b003c297b914cb08f737b1f4d40b16d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:51.154736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b003c297b914cb08f737b1f4d40b16d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:51.156947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:51.183399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=3,keystone:DELETE=7,keystone:SELECT=90 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:51.215043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b003c297b914cb08f737b1f4d40b16d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.229266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b003c297b914cb08f737b1f4d40b16d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.230027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b003c297b914cb08f737b1f4d40b16d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.230658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b003c297b914cb08f737b1f4d40b16d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.238331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b003c297b914cb08f737b1f4d40b16d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.256994 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1278/2571] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:51 2022] DELETE /identity/v3/users/3b003c297b914cb08f737b1f4d40b16d => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:51.437385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:51.457490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a7caeb3-9f5d-47c2-821c-d4ddf439ddf4 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:51.483251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:51.526290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:51.526929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:51.527208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:51.527511 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] PATH_INFO: `/v3/projects/ea0d7081bf384d098fad20ef507f4dfd` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:51.528669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea0d7081bf384d098fad20ef507f4dfd)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:51.530553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:51.576103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea0d7081bf384d098fad20ef507f4dfd', 'request_id': 'req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.584909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea0d7081bf384d098fad20ef507f4dfd', 'request_id': 'req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.591505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea0d7081bf384d098fad20ef507f4dfd', 'request_id': 'req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.601671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] The token cache is being invalidate because project ea0d7081bf384d098fad20ef507f4dfd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:51.601960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.602610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.604171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1294/2572] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:51 2022] DELETE /identity/v3/projects/ea0d7081bf384d098fad20ef507f4dfd => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:51.608299 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:51.609700 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:51.622474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a369467-53f0-4b68-967d-de54506f732b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:51.679514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:51.680157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:51.680441 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:51.680736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] PATH_INFO: `/v3/users/76a3927485aa4ebca964a71fdbb16a87` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:51.681904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76a3927485aa4ebca964a71fdbb16a87)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:51.683878 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:51.736737 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76a3927485aa4ebca964a71fdbb16a87'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.750733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76a3927485aa4ebca964a71fdbb16a87'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.751721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76a3927485aa4ebca964a71fdbb16a87'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.753014 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76a3927485aa4ebca964a71fdbb16a87'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.760667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76a3927485aa4ebca964a71fdbb16a87'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:51.778023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1279/2573] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:51 2022] DELETE /identity/v3/users/76a3927485aa4ebca964a71fdbb16a87 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:28:52.279776 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:52.281214 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:52.294280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d0010-bb70-44b6-bde0-4fa2d88a97f5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:52.350739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:52.351389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:52.351715 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:52.351992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] PATH_INFO: `/v3/users/edfb7c31479e4cc6bd228af8e93b3dfa` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:52.353165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] RBAC: Authorizing `identity:delete_user(user_id=edfb7c31479e4cc6bd228af8e93b3dfa)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:52.355151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:52.404218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'edfb7c31479e4cc6bd228af8e93b3dfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.411562 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:52.414240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:52.417238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'edfb7c31479e4cc6bd228af8e93b3dfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.417956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'edfb7c31479e4cc6bd228af8e93b3dfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.419838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:52.438352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'edfb7c31479e4cc6bd228af8e93b3dfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.439869 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d3f01542-bc1a-4312-97f2-4610e4e0e94c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'edfb7c31479e4cc6bd228af8e93b3dfa'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.440443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e04a18c-5ee8-430e-92a2-cf6951b8eab0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:52.446504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:52.447168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:52.447489 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:52.447922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] PATH_INFO: `/v3/projects/66b6d76f31f14cd2afd43cc377c35714` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:52.449505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] RBAC: Authorizing `identity:delete_project(project_id=66b6d76f31f14cd2afd43cc377c35714)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:52.451781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:52.457991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1295/2574] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:52 2022] DELETE /identity/v3/users/edfb7c31479e4cc6bd228af8e93b3dfa => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:52.462388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:52.462755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:52.463105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:52.469555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:52.483659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:52.485147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:52.514326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66b6d76f31f14cd2afd43cc377c35714', 'request_id': 'req-c397b43a-1803-482c-9809-67e9e74fd658'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.526483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66b6d76f31f14cd2afd43cc377c35714', 'request_id': 'req-c397b43a-1803-482c-9809-67e9e74fd658'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.537575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66b6d76f31f14cd2afd43cc377c35714', 'request_id': 'req-c397b43a-1803-482c-9809-67e9e74fd658'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.550211 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] The token cache is being invalidate because project 66b6d76f31f14cd2afd43cc377c35714 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:52.550496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.551109 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:52.552864 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1280/2575] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:52 2022] DELETE /identity/v3/projects/66b6d76f31f14cd2afd43cc377c35714 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:52.554407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1296/2576] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:52.568791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:52.570815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:52.643963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c397b43a-1803-482c-9809-67e9e74fd658 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:52.649142 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:52.649796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:52.650074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:52.650344 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:52.652272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:52.729408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:52.730907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f32908-128e-482f-abb5-851f7506167e service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:52.743041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1281/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:53.063471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.068497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.081992 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f60418d-a632-40db-9f11-0b990d3c603b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.087240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.088043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.089250 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.089868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] PATH_INFO: `/v3/projects/857f68541e904e7abd38151d6c72d687` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.091868 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] RBAC: Authorizing `identity:delete_project(project_id=857f68541e904e7abd38151d6c72d687)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.094728 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.150733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.151209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.151592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.162953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '857f68541e904e7abd38151d6c72d687', 'request_id': 'req-fc930959-ab2b-41c8-9e54-32498bc7c528'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:53.172255 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '857f68541e904e7abd38151d6c72d687', 'request_id': 'req-fc930959-ab2b-41c8-9e54-32498bc7c528'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:53.175018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:53.176273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.182571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '857f68541e904e7abd38151d6c72d687', 'request_id': 'req-fc930959-ab2b-41c8-9e54-32498bc7c528'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:53.194896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] The token cache is being invalidate because project 857f68541e904e7abd38151d6c72d687 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:53.195191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:53.195926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-fc930959-ab2b-41c8-9e54-32498bc7c528 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:53.197607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1297/2578] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:53 2022] DELETE /identity/v3/projects/857f68541e904e7abd38151d6c72d687 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:53.203478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c9a9-9f07-463c-a721-d057de5623d7 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.203770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c9a9-9f07-463c-a721-d057de5623d7 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.204053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59c9a9-9f07-463c-a721-d057de5623d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.224162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-1c59c9a9-9f07-463c-a721-d057de5623d7 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:53.225386 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1c59c9a9-9f07-463c-a721-d057de5623d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.232507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:53.245204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1282/2579] 10.222.0.40 () {58 vars in 1009 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:53.259073 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.260528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.286856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:53.302968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1298/2580] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:28:53.309227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.309602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.309956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.331062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:53.332317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.340486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1299/2581] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:53.344458 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.346409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbd9ad8-39e6-44f2-b2ab-d7193904621c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.352713 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.353380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.353668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.354081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.355210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83667a94-d838-45f2-981c-83edc9e129fc None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.361569 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.362194 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.362480 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.362751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.363409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.363547 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:28:53.363797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.365141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.365471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.375680 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1283/2582] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:28:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 31 14:28:53.377492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1300/2583] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:28:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:28:53.381588 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.391902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ccc458-de99-4af2-9639-977447762e6e service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.397609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.398232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.398519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.398787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.399481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.401559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.433550 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1284/2584] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:53.437736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.448168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc41008-fa94-4bd8-bf12-dcf649d7a048 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.517373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.518791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.519156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.519523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.520365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.522884 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.556232 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1301/2585] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/users => generated 287 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:28:53.557499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.569373 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0127841-5115-4cf4-ac61-ab8a41da092f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.575009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.576136 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.576625 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.577098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.578010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.579850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.593149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1285/2586] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:28:53.597913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.610953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e1fbab-4f2a-4759-a0e1-dd4c56a8e4e0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.617476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.618311 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.618598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.618881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.619840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.621834 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a54627ad-831e-4ed9-b351-bc4e417d9ff6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.634116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1302/2587] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:28:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:28:53.638452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.649866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc84314f-b26b-4507-86e1-bfea0315930d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.654594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.655280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.655599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.655876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] PATH_INFO: `/v3/projects/37fd7aaf529d445a9c9ab025747dcfd5/users/2b2b0ae697fa44f3b05c6be0d051bbc2/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.675253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] RBAC: Authorizing `identity:create_grant(project_id=37fd7aaf529d445a9c9ab025747dcfd5, user_id=2b2b0ae697fa44f3b05c6be0d051bbc2, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.678291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.696952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1286/2588] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:28:53 2022] PUT /identity/v3/projects/37fd7aaf529d445a9c9ab025747dcfd5/users/2b2b0ae697fa44f3b05c6be0d051bbc2/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:28:53.712498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.712787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.713066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.757640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] MFA Rules not processed for user `2b2b0ae697fa44f3b05c6be0d051bbc2`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:53.759514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.840133 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1303/2589] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:53 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:53.847447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:53.849401 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:53.917481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc3512c-1815-4eb7-8b7c-c4127c79efdc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:53.921212 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:53.921835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:53.922115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:53.922389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:53.931158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:53.932889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beacf27a-af92-47b4-ae79-d427104a7ec0 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:53.942834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1287/2590] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:54.036789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:54.088826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:54.110805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3d48728-865e-4f3f-a7cc-3a3d9e10ec98 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:54.116151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:54.116840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:54.117120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:54.117396 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] PATH_INFO: `/v3/users/2b2b0ae697fa44f3b05c6be0d051bbc2` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:54.118567 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b2b0ae697fa44f3b05c6be0d051bbc2)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:54.120877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:54.193875 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b2b0ae697fa44f3b05c6be0d051bbc2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.212826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b2b0ae697fa44f3b05c6be0d051bbc2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.213529 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b2b0ae697fa44f3b05c6be0d051bbc2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.222430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b2b0ae697fa44f3b05c6be0d051bbc2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.223952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b2b0ae697fa44f3b05c6be0d051bbc2'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.245872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1304/2591] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:28:54 2022] DELETE /identity/v3/users/2b2b0ae697fa44f3b05c6be0d051bbc2 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:28:54.250688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:54.250978 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:54.251259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:54.278549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:28:54.279797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:54.323214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:28:54.365310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1288/2592] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:28:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:28:54.374818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:54.376384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:54.436988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28884f78-e5ed-4b99-ad9c-081f6f0a3234 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:54.441747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:54.442484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:54.442831 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:54.443172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:54.453549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:54.455116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c16f1b9-a839-44df-b1f9-fc9ab3d204c0 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:54.465376 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1305/2593] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:28:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:28:54.751846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:28:54.753563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:28:54.777406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0407d64e-0818-4ab7-b4c3-5323251cb18f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:28:54.792052 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:28:54.792729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:28:54.793004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:28:54.793282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] PATH_INFO: `/v3/projects/37fd7aaf529d445a9c9ab025747dcfd5` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:28:54.796386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=37fd7aaf529d445a9c9ab025747dcfd5)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:28:54.798442 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:28:54.850267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37fd7aaf529d445a9c9ab025747dcfd5', 'request_id': 'req-58f74386-ef0c-4548-a27b-eb85eba6d1ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.856746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37fd7aaf529d445a9c9ab025747dcfd5', 'request_id': 'req-58f74386-ef0c-4548-a27b-eb85eba6d1ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.865375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37fd7aaf529d445a9c9ab025747dcfd5', 'request_id': 'req-58f74386-ef0c-4548-a27b-eb85eba6d1ef'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.881999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] The token cache is being invalidate because project 37fd7aaf529d445a9c9ab025747dcfd5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:28:54.882282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.882949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:28:54.884801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1289/2594] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:28:54 2022] DELETE /identity/v3/projects/37fd7aaf529d445a9c9ab025747dcfd5 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:04.460724 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=60,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:04.878770 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=39,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:09.968156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:09.968622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:09.968729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:09.988337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] MFA Rules not processed for user `eb9910cbfb414432b3425c7dfdf79787`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:09.989563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:10.045299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1306/2595] 10.222.0.40 () {58 vars in 967 bytes} [Sun Jul 31 14:29:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:10.057417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:10.058816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:10.132421 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f74386-ef0c-4548-a27b-eb85eba6d1ef admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:10.137606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d498354314244afa4445a1ed99d15cf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:10.138263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:10.138544 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:10.138822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:10.147997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:10.149624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:10.158697 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1290/2596] 10.222.0.40 () {60 vars in 1384 bytes} [Sun Jul 31 14:29:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 31 14:29:20.037057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=25 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:20.154664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=32 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:21.445059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:21.446486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:21.501894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a08056-a38e-4b64-a08c-7b9670af07b8 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:21.505980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:21.506611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:21.506877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:21.507154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:21.508920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:21.574157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:21.575896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:21.585977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1307/2597] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:24.739947 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:24.741940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:24.755832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af4c8ee8-5950-4101-97a3-ec393208e5e9 service swift] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:24.760413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:24.761057 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:24.761338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:24.761613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] PATH_INFO: `/v3/users/d6e8399080a942f1b689fc8c6b42c717` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:24.762809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6e8399080a942f1b689fc8c6b42c717)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:24.764739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:24.814599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6e8399080a942f1b689fc8c6b42c717'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:24.828098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6e8399080a942f1b689fc8c6b42c717'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:24.828792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6e8399080a942f1b689fc8c6b42c717'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:24.829594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6e8399080a942f1b689fc8c6b42c717'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:24.837233 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-20528fbb-29e5-4693-b525-dedf641056f7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6e8399080a942f1b689fc8c6b42c717'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:24.850161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1291/2598] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:24 2022] DELETE /identity/v3/users/d6e8399080a942f1b689fc8c6b42c717 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:29:25.031224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.042864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38f3b1b-3671-433f-b01a-973379af8b98 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.100952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.101595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.101876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.102152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] PATH_INFO: `/v3/projects/9097fd66951e457ba46812ff17db7501` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.103304 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=9097fd66951e457ba46812ff17db7501)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.105259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.150972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9097fd66951e457ba46812ff17db7501', 'request_id': 'req-98d165f2-200a-4fa0-853f-f5d052966e1b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:25.153434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:25.159126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9097fd66951e457ba46812ff17db7501', 'request_id': 'req-98d165f2-200a-4fa0-853f-f5d052966e1b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:25.166135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9097fd66951e457ba46812ff17db7501', 'request_id': 'req-98d165f2-200a-4fa0-853f-f5d052966e1b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:25.177384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] The token cache is being invalidate because project 9097fd66951e457ba46812ff17db7501 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:25.177674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:25.178312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-98d165f2-200a-4fa0-853f-f5d052966e1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:25.179873 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1308/2599] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:25 2022] DELETE /identity/v3/projects/9097fd66951e457ba46812ff17db7501 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:25.184801 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.185098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.185406 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.204765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:25.205963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:25.266920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1292/2600] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:25.274933 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.275154 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.275450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.296792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:25.298110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:25.306945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1309/2601] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:25.311448 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.321323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ee8900-3fa7-42bf-9959-3202bffcd65f None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.326103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.326742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.327021 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.327294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.328040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:29:25.328353 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.329927 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.347634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1293/2602] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:29:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:29:25.348752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.358021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1046a0-9dc6-4f5b-8e16-3c320036ad45 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.363283 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.363954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.364237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.364509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.365179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.367213 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.394450 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1310/2603] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/projects => generated 403 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:29:25.398529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.408238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bd09f5-fea7-47e8-9556-e14f89016ae7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.463044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.463667 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.463942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.464208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.464865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.466778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.474915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:25.489514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1294/2604] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/users => generated 302 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:29:25.493536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.503540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cedf83cd-3cf7-4a54-8365-d23d68a4012c admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.508020 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.508615 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.508898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.509169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.509859 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.511448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.521784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1311/2605] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:25.525677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.535881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b0e46-5e8b-4f6d-98f1-91a7801f57da admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.540465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.541065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.541341 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.541614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.542301 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.543916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c59e5e-5089-4bf2-9a33-daaac2ebf75f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.554160 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1295/2606] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:25.558354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.568350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84e9d5c-0a87-4851-937f-a21222bfe37f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.572945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.573664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.573878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.574157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] PATH_INFO: `/v3/projects/0b250eb65d9145aa80415c7302610f27/users/24c7b47c5dfe4e479821bad34a9d035c/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.588979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b250eb65d9145aa80415c7302610f27, user_id=24c7b47c5dfe4e479821bad34a9d035c, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.591627 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.605468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1312/2607] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:25 2022] PUT /identity/v3/projects/0b250eb65d9145aa80415c7302610f27/users/24c7b47c5dfe4e479821bad34a9d035c/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:25.615636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.615922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.616198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.643746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] MFA Rules not processed for user `24c7b47c5dfe4e479821bad34a9d035c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:25.644985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:25.708416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1296/2608] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:25.713041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.773695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d03638-e97f-40e8-80d0-ad2f043e55e6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.778593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.779165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.779500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.779777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.780449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.782366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.808778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1313/2609] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/projects => generated 403 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:25.812820 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.823314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f615e01-67ee-4f3c-aad8-eb0606d923cc None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.879370 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.880009 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.880288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.880559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.881213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.883117 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.904546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1297/2610] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:25 2022] POST /identity/v3/users => generated 308 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:25.908707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.919614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ec1e6c-06ab-4e37-b2b7-d564273f473e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.924507 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.925112 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.925395 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.925672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.926366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.928000 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.938346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1314/2611] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:25.942918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:25.953694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b2e0c7-2b8f-4a32-b505-49512bcb427a admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:25.958979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:25.959690 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:25.959976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:25.960255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] PATH_INFO: `/v3/projects/72df7af7c8d743c8a44ba493a092bb18/users/f82674ef594646469a5970c62b2c4e22/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:25.976773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df7af7c8d743c8a44ba493a092bb18, user_id=f82674ef594646469a5970c62b2c4e22, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:25.979499 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:25.992937 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1298/2612] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:25 2022] PUT /identity/v3/projects/72df7af7c8d743c8a44ba493a092bb18/users/f82674ef594646469a5970c62b2c4e22/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:25.996936 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:26.008163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954cef1d-80bd-4284-8fed-edebe9b147cd admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:26.055991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:26.069565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:26.070191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.070475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.070756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.071491 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:26.073129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6763780c-03b5-4d27-95fa-e3702fefb896 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:26.084033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1315/2613] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:25 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:26.088205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:26.098482 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a102338-f05a-4d43-9b00-78d7204781c4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:26.103055 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:26.103748 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.104024 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.104298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] PATH_INFO: `/v3/projects/72df7af7c8d743c8a44ba493a092bb18/users/f82674ef594646469a5970c62b2c4e22/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.106337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=72df7af7c8d743c8a44ba493a092bb18, user_id=f82674ef594646469a5970c62b2c4e22, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:26.108995 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:26.122078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1299/2614] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:26 2022] PUT /identity/v3/projects/72df7af7c8d743c8a44ba493a092bb18/users/f82674ef594646469a5970c62b2c4e22/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:26.134120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.134405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.134684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.162674 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] MFA Rules not processed for user `f82674ef594646469a5970c62b2c4e22`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:26.163917 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:26.229308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1316/2615] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:26 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:29:26.240080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:26.241781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:26.264089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:26.299779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c078964b-49bd-4d97-9b3e-48596e9bb2ce admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:26.303415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:26.304053 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.304333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.304605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.313526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:26.315581 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b418818-9747-43bc-b251-96ef355229b7 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:26.325483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1300/2616] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:26 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:26.470032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:26.530328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b853539-9e06-48c8-9905-12b7fef77e26 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:26.534907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:26.535586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.535845 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.536125 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] PATH_INFO: `/v3/users/24c7b47c5dfe4e479821bad34a9d035c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.537282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24c7b47c5dfe4e479821bad34a9d035c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:26.539205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:26.584815 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24c7b47c5dfe4e479821bad34a9d035c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:26.596263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24c7b47c5dfe4e479821bad34a9d035c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:26.596915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24c7b47c5dfe4e479821bad34a9d035c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:26.603956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24c7b47c5dfe4e479821bad34a9d035c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:26.604566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24c7b47c5dfe4e479821bad34a9d035c'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:26.606544 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:26.619307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1317/2617] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:26 2022] DELETE /identity/v3/users/24c7b47c5dfe4e479821bad34a9d035c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:26.623952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.624236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.624508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.643831 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:26.645203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:26.708295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1301/2618] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:26.718104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:26.719581 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:26.779367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986025cf-c59b-4aa4-ba40-73faf8177d9b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:26.783040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:26.784193 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:26.784472 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:26.784747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:26.793641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:26.795066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:26.805028 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1318/2619] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:27.074013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.075736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:27.088603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70810dd3-af2d-4402-8457-1a93232e202c None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:27.093158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:27.094044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.094322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.094601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] PATH_INFO: `/v3/projects/0b250eb65d9145aa80415c7302610f27` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.095787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b250eb65d9145aa80415c7302610f27)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:27.099012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:27.151310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b250eb65d9145aa80415c7302610f27', 'request_id': 'req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.157293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b250eb65d9145aa80415c7302610f27', 'request_id': 'req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.165351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b250eb65d9145aa80415c7302610f27', 'request_id': 'req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.175696 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] The token cache is being invalidate because project 0b250eb65d9145aa80415c7302610f27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:27.175983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.176600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.178104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1302/2620] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:27 2022] DELETE /identity/v3/projects/0b250eb65d9145aa80415c7302610f27 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:27.182070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.192195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de228bf7-e05d-440e-981f-602c95cf2324 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:27.247753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:27.248393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.248675 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.248954 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] PATH_INFO: `/v3/users/f82674ef594646469a5970c62b2c4e22` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.250116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=f82674ef594646469a5970c62b2c4e22)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:27.252064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:27.297677 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f82674ef594646469a5970c62b2c4e22'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.309207 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f82674ef594646469a5970c62b2c4e22'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.309874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f82674ef594646469a5970c62b2c4e22'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.317044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f82674ef594646469a5970c62b2c4e22'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.317672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ad999661-07eb-4fe0-b823-35db5a617bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f82674ef594646469a5970c62b2c4e22'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.330104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1319/2621] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:27 2022] DELETE /identity/v3/users/f82674ef594646469a5970c62b2c4e22 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:27.596894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.598338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:27.611126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d84570f-c3c3-4ff0-819a-9c04c391bd7d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:27.656909 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:27.671031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:27.671711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.671992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.672266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] PATH_INFO: `/v3/projects/72df7af7c8d743c8a44ba493a092bb18` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.673454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72df7af7c8d743c8a44ba493a092bb18)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:27.675332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:27.720001 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72df7af7c8d743c8a44ba493a092bb18', 'request_id': 'req-7aebf342-047f-424f-ac73-f23f8ed2d6b1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.725828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72df7af7c8d743c8a44ba493a092bb18', 'request_id': 'req-7aebf342-047f-424f-ac73-f23f8ed2d6b1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.732838 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72df7af7c8d743c8a44ba493a092bb18', 'request_id': 'req-7aebf342-047f-424f-ac73-f23f8ed2d6b1'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.743348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] The token cache is being invalidate because project 72df7af7c8d743c8a44ba493a092bb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:27.743668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.744282 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-7aebf342-047f-424f-ac73-f23f8ed2d6b1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:27.745769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1303/2622] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:27 2022] DELETE /identity/v3/projects/72df7af7c8d743c8a44ba493a092bb18 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:27.750771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.751105 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.751375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.772602 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:27.773804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:27.781971 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:27.832746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1320/2623] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:27.840428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.840707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.840977 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.860355 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:27.861557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:27.870352 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1304/2624] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:27.874514 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.884770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30b1b60-6d8c-4b4b-a4cc-50375e00c6ab None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:27.889267 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:27.889891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.890167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.890452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.891123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:29:27.891440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:27.893043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:27.904756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1321/2625] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:29:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:29:27.908993 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.918141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcab6c67-6fda-4258-a712-3ee4305c5fca None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:27.922790 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:27.923388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:27.923749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:27.923970 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:27.925115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:27.926990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:27.953832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1305/2626] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:27 2022] POST /identity/v3/projects => generated 403 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:27.957858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:27.967620 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3554884b-7bbe-4dc7-9edf-81e264e616db admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:28.021492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:28.022090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.022365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.022634 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.023281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:28.025201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:28.046478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1322/2627] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:27 2022] POST /identity/v3/users => generated 302 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:29:28.051222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:28.061425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ab26b0-2b86-4eb1-9bf4-c0d5e4379bdf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:28.066714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:28.067323 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.067658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.067935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.068633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:28.070225 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:28.080274 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1306/2628] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:28.084225 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:28.093945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e381497d-b673-46bd-816e-49643e1b0683 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:28.098488 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:28.099084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.099363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.099667 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.100347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:28.101919 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06dd85e-9337-4672-bc04-25e51dcbe125 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:28.112699 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1323/2629] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:28.116716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:28.126660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2c5b05-a40a-4cbd-9c69-62c19bbf9e54 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:28.132706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:28.133603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.134082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.134557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] PATH_INFO: `/v3/projects/07c8eab62c8546c1afe905f01588fa54/users/af209a4eb00a47dabf8a4379c6689e6d/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.149832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] RBAC: Authorizing `identity:create_grant(project_id=07c8eab62c8546c1afe905f01588fa54, user_id=af209a4eb00a47dabf8a4379c6689e6d, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:28.152505 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:28.166015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1307/2630] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:28 2022] PUT /identity/v3/projects/07c8eab62c8546c1afe905f01588fa54/users/af209a4eb00a47dabf8a4379c6689e6d/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:29:28.171012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d0ce-6140-4e61-851a-951cab80009d None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.171298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d0ce-6140-4e61-851a-951cab80009d None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.171606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3654d0ce-6140-4e61-851a-951cab80009d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.190730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3654d0ce-6140-4e61-851a-951cab80009d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:28.191891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3654d0ce-6140-4e61-851a-951cab80009d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:28.248732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1324/2631] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:29:28.258393 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:28.260130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:28.316265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc7cc67-f050-49f6-978a-86dbfea76baf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:28.319853 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:28.320467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.320747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.321022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.330133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:28.331612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:28.340910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1308/2632] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:28.959346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:28.959664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:28.959940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:28.987523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] MFA Rules not processed for user `af209a4eb00a47dabf8a4379c6689e6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:28.988733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:28.997446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:29.059909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1325/2633] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:29.064293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.076152 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2f1460-a585-46e6-bc9a-47c338e8b9de service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.081865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.082566 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.082910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.083250 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.084038 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.086000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.100517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:29.119061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1309/2634] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:29 2022] POST /identity/v3/projects => generated 403 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:29.120696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.131006 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.133034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:29.133561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c192c7-644a-4d4e-a8b2-17960a0e972e None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.196665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4826535a-ce6b-40ff-8762-ed19d8fe5c30 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.198734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.199362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.199676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.199981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.200657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.200923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.201267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.201665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.201945 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.202927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.203807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:29.225714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1326/2635] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:29 2022] POST /identity/v3/users => generated 308 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:29.229393 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.239005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b5c176-3300-442e-b444-03de4a08e6aa admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.245010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.245626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.245912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.246238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.247373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.249394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.260258 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1327/2636] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:29:29.264477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.266362 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.267902 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.274320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bfcb20-ba30-4e97-b36c-d5f705de24a5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.277366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1310/2637] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:29 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:29.279116 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.279813 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.280095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.280373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] PATH_INFO: `/v3/projects/fdbd92e40b79426787af1b46f4d6337d/users/882165e382754299bd9db495a2789def/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.295282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbd92e40b79426787af1b46f4d6337d, user_id=882165e382754299bd9db495a2789def, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.297962 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.312110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1328/2638] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:29 2022] PUT /identity/v3/projects/fdbd92e40b79426787af1b46f4d6337d/users/882165e382754299bd9db495a2789def/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:29.316234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.327332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312474e0-c395-4c11-8563-6c3bae819268 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.393922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.394657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.395000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.395278 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.396000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.397605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11591f1c-6d29-4127-8da5-558a2add5211 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.407620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1311/2639] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:29 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:29:29.411730 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:29.421826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc39d30-1033-49c4-9159-b788c6bd98c5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:29.426896 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:29.427714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:29.427991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:29.428265 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] PATH_INFO: `/v3/projects/fdbd92e40b79426787af1b46f4d6337d/users/882165e382754299bd9db495a2789def/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:29.430224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdbd92e40b79426787af1b46f4d6337d, user_id=882165e382754299bd9db495a2789def, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:29.432871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:29.444391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1329/2640] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:29 2022] PUT /identity/v3/projects/fdbd92e40b79426787af1b46f4d6337d/users/882165e382754299bd9db495a2789def/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:30.082456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:30.083003 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:30.083121 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:30.114412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] MFA Rules not processed for user `882165e382754299bd9db495a2789def`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:30.115915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:30.181242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:30.239949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1312/2641] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:30 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:31.411811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:31.466050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:31.478806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c31ce5-f183-4a04-8970-210694895546 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:31.483334 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:31.484013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:31.484295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:31.484571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] PATH_INFO: `/v3/users/af209a4eb00a47dabf8a4379c6689e6d` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:31.485725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=af209a4eb00a47dabf8a4379c6689e6d)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:31.488281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:31.539771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af209a4eb00a47dabf8a4379c6689e6d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:31.552540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af209a4eb00a47dabf8a4379c6689e6d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:31.553262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af209a4eb00a47dabf8a4379c6689e6d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:31.561251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af209a4eb00a47dabf8a4379c6689e6d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:31.561901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af209a4eb00a47dabf8a4379c6689e6d'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:31.581367 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1330/2642] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:31 2022] DELETE /identity/v3/users/af209a4eb00a47dabf8a4379c6689e6d => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:29:32.013750 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:32.025829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194ab55a-93a1-4278-97ce-1ceca3452732 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:32.092426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:32.093627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.093922 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.094201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] PATH_INFO: `/v3/projects/07c8eab62c8546c1afe905f01588fa54` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.095375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] RBAC: Authorizing `identity:delete_project(project_id=07c8eab62c8546c1afe905f01588fa54)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:32.097414 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:32.152025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '07c8eab62c8546c1afe905f01588fa54', 'request_id': 'req-d0fd1182-67e3-463d-8bea-5682d0721c2b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.158576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '07c8eab62c8546c1afe905f01588fa54', 'request_id': 'req-d0fd1182-67e3-463d-8bea-5682d0721c2b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.167607 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '07c8eab62c8546c1afe905f01588fa54', 'request_id': 'req-d0fd1182-67e3-463d-8bea-5682d0721c2b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.179479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] The token cache is being invalidate because project 07c8eab62c8546c1afe905f01588fa54 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:32.179765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.180394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.181906 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1313/2643] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:32 2022] DELETE /identity/v3/projects/07c8eab62c8546c1afe905f01588fa54 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:32.185826 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:32.187224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:32.201392 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9981c72-b07b-44f3-bac6-7e885be3a0ac admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:32.383880 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:32.384671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.385022 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.385299 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] PATH_INFO: `/v3/users/882165e382754299bd9db495a2789def` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.386475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] RBAC: Authorizing `identity:delete_user(user_id=882165e382754299bd9db495a2789def)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:32.388448 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:32.435402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882165e382754299bd9db495a2789def'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.449734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882165e382754299bd9db495a2789def'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.450705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882165e382754299bd9db495a2789def'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.458987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882165e382754299bd9db495a2789def'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.459756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-7bf3c42d-e3cb-431e-93d0-658fc4180cae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882165e382754299bd9db495a2789def'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.461036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:32.474341 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1331/2644] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:32 2022] DELETE /identity/v3/users/882165e382754299bd9db495a2789def => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:32.656899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:32.659087 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:32.674556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fd1182-67e3-463d-8bea-5682d0721c2b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:32.751943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:32.752778 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.753170 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.753531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] PATH_INFO: `/v3/projects/fdbd92e40b79426787af1b46f4d6337d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.755105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdbd92e40b79426787af1b46f4d6337d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:32.757322 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:32.759187 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:32.809135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdbd92e40b79426787af1b46f4d6337d', 'request_id': 'req-ee41d942-e720-4d05-82b9-7b36f29ea555'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.815465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdbd92e40b79426787af1b46f4d6337d', 'request_id': 'req-ee41d942-e720-4d05-82b9-7b36f29ea555'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.823548 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdbd92e40b79426787af1b46f4d6337d', 'request_id': 'req-ee41d942-e720-4d05-82b9-7b36f29ea555'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.834624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] The token cache is being invalidate because project fdbd92e40b79426787af1b46f4d6337d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:32.834908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.835605 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ee41d942-e720-4d05-82b9-7b36f29ea555 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:32.837171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1314/2645] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:32 2022] DELETE /identity/v3/projects/fdbd92e40b79426787af1b46f4d6337d => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:32.842130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.842416 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.842693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.863062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:32.864293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:32.921370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1332/2646] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:32.929780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.930061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.930332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.949875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:32.951262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:32.959521 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1315/2647] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:32.963722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:32.973769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fb1080-1b08-4fb1-b63e-0c3fcfbecc29 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:32.978307 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:32.978910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:32.979187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:32.979482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:32.980202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:29:32.980506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:32.982095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:32.994695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1333/2648] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:29:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:29:32.998954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.008408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6c35cb-87ba-4fe4-a0c8-94d80bfbdf56 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.014717 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.016321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.016674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.017016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.018266 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.020865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.052597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1316/2649] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:32 2022] POST /identity/v3/projects => generated 377 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:33.056908 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.067070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71363da9-11fd-4318-a771-f19db005a398 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.134339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.135015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.135372 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.135796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.136608 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.138424 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.179624 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1334/2650] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:33 2022] POST /identity/v3/users => generated 289 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:33.181069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.196083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e53f16-a78d-46e6-8edb-02cdff3b5179 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.203739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.205990 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.206490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.206929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.212476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.215934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.227459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1317/2651] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:33.231352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.241362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a6aa03-2314-4061-a96d-b750ba23b02e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.246041 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.246649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.246927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.247199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.248010 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.249763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c60f9f7-e792-4305-be03-d0b315bd9f04 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.260168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1335/2652] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:33.264484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.275879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cfc9ac-0e4d-450a-bf52-75917fea2269 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.280711 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.281453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.281806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.282165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] PATH_INFO: `/v3/projects/c71eeab9c6314f68b5c7028be35e5134/users/2e1287ffa55241d78592e098c09d4f83/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.297302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c71eeab9c6314f68b5c7028be35e5134, user_id=2e1287ffa55241d78592e098c09d4f83, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.299999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.312498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1318/2653] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:33 2022] PUT /identity/v3/projects/c71eeab9c6314f68b5c7028be35e5134/users/2e1287ffa55241d78592e098c09d4f83/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:33.323752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4472121-871b-4218-8e67-37433774aaff None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.324045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4472121-871b-4218-8e67-37433774aaff None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.324323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4472121-871b-4218-8e67-37433774aaff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.352951 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d4472121-871b-4218-8e67-37433774aaff None None] MFA Rules not processed for user `2e1287ffa55241d78592e098c09d4f83`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:33.354167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d4472121-871b-4218-8e67-37433774aaff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:33.400106 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:33.423057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1336/2654] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:33.428231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.493044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839eb25a-7a4c-4074-9172-c230c3550860 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.498819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.499457 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.499741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.500013 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.500674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.503097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.530148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1319/2655] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:29:33 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:33.534189 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4472121-871b-4218-8e67-37433774aaff None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.545173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4472121-871b-4218-8e67-37433774aaff None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.604170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.604798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.605077 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.605347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.606038 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.607977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.630914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1337/2656] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:29:33 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:29:33.634788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.645767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fb7be4-56f7-43a6-913f-a4b089ec1b8e admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.650845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.651522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.651809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.652083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.652793 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.654417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.660395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:33.666980 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1320/2657] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:33.671167 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.681690 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cb008d-2e04-4c95-a6d3-90a6732280d0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.686611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.687282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.687594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.687872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] PATH_INFO: `/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3/users/a8d681b0607b4fc3a4f17875bd3a874b/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.704252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57eb00bd0d64381b63ecf04475cb0f3, user_id=a8d681b0607b4fc3a4f17875bd3a874b, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.707756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.721049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1338/2658] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:33 2022] PUT /identity/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3/users/a8d681b0607b4fc3a4f17875bd3a874b/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:33.725089 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.735040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd0cd52-38d7-44c5-94d4-2c1d57361fa5 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.790804 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.791365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.791674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.791948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.792640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.794254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5d9ab5-bc70-498a-97df-72b25fd69d6d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.804241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1321/2659] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:29:33 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:29:33.808331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.819044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47804ce6-3517-45af-a3f9-02eb95227ba8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:33.823683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:33.824339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.824619 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.824920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] PATH_INFO: `/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3/users/a8d681b0607b4fc3a4f17875bd3a874b/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.827088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=d57eb00bd0d64381b63ecf04475cb0f3, user_id=a8d681b0607b4fc3a4f17875bd3a874b, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:33.829769 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:33.839599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1339/2660] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:29:33 2022] PUT /identity/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3/users/a8d681b0607b4fc3a4f17875bd3a874b/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:33.849668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:33.849954 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:33.850234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:33.878619 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] MFA Rules not processed for user `a8d681b0607b4fc3a4f17875bd3a874b`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:29:33.879889 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:33.947603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1322/2661] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:29:33 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:29:33.958470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:33.960622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:34.018091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b663994-ae23-4757-a0c5-b772f9eb50ec admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:34.021656 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:34.022256 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:34.022531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:34.022808 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:34.071800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:34.091168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:34.093126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:34.102115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1340/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:33 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:34.519865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:34.521437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:34.583659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f3810-6662-41d7-8d28-6f51ba86ad09 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:34.587108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:34.587919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:34.588279 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:34.588561 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:34.590350 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:34.627368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:34.657603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:34.659822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:34.669781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1323/2663] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:38.776984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:38.786224 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad0ae1b-bda7-4022-a7d7-69dbc88b85ee service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:38.789921 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:38.790540 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:38.790820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:38.791093 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:38.801280 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:38.802711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:38.812864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1341/2664] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:38 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:29:40.747651 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:40.749241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:40.777413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df22986-b03f-4484-86f9-c40de018e66a service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:40.783110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:40.783789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:40.784071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:40.784349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] PATH_INFO: `/v3/users/26ca8614358245b99f621edc7f1d0021` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:40.786602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=26ca8614358245b99f621edc7f1d0021)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:40.788575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:40.835257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '26ca8614358245b99f621edc7f1d0021'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:40.847318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '26ca8614358245b99f621edc7f1d0021'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:40.848005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '26ca8614358245b99f621edc7f1d0021'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:40.848606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '26ca8614358245b99f621edc7f1d0021'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:40.855315 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '26ca8614358245b99f621edc7f1d0021'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:40.868190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1324/2665] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:40 2022] DELETE /identity/v3/users/26ca8614358245b99f621edc7f1d0021 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:41.038894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:41.049604 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18c85e1-70d0-4010-8c0b-62ed40b88219 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:41.106477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:41.107040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:41.107318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:41.107623 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] PATH_INFO: `/v3/projects/d9ff0edf56fa403e8b9bf25fe004cdd3` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:41.108765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ff0edf56fa403e8b9bf25fe004cdd3)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:41.110648 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:41.156846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ff0edf56fa403e8b9bf25fe004cdd3', 'request_id': 'req-c15170f2-bcc3-40cc-8c31-b149c5d1a438'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.170526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ff0edf56fa403e8b9bf25fe004cdd3', 'request_id': 'req-c15170f2-bcc3-40cc-8c31-b149c5d1a438'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.177219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ff0edf56fa403e8b9bf25fe004cdd3', 'request_id': 'req-c15170f2-bcc3-40cc-8c31-b149c5d1a438'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.187455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] The token cache is being invalidate because project d9ff0edf56fa403e8b9bf25fe004cdd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:41.187780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.188433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.190800 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1342/2666] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:41 2022] DELETE /identity/v3/projects/d9ff0edf56fa403e8b9bf25fe004cdd3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:41.194877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:41.196525 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:41.211000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47d7e19-8e58-4382-97f0-ed865148ee5d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:41.265126 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:41.265781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:41.266063 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:41.266360 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] PATH_INFO: `/v3/users/2a8c741e516c43b1844320c9f8b0e7ff` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:41.268191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a8c741e516c43b1844320c9f8b0e7ff)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:41.270097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:41.312817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:41.315045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a8c741e516c43b1844320c9f8b0e7ff'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.328615 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a8c741e516c43b1844320c9f8b0e7ff'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.329296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a8c741e516c43b1844320c9f8b0e7ff'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.329894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a8c741e516c43b1844320c9f8b0e7ff'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.336633 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a8c741e516c43b1844320c9f8b0e7ff'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.350542 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1325/2667] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:29:41 2022] DELETE /identity/v3/users/2a8c741e516c43b1844320c9f8b0e7ff => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:41.528791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:41.530262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:41.542909 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15170f2-bcc3-40cc-8c31-b149c5d1a438 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:41.601500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:41.602145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:41.602429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:41.602697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] PATH_INFO: `/v3/projects/a062733c92a64fd68acb9a59fbba8b24` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:41.603874 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a062733c92a64fd68acb9a59fbba8b24)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:41.605789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:41.616477 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:41.652192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a062733c92a64fd68acb9a59fbba8b24', 'request_id': 'req-e5180876-984a-4619-a3c2-f639af6985a5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.660161 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a062733c92a64fd68acb9a59fbba8b24', 'request_id': 'req-e5180876-984a-4619-a3c2-f639af6985a5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.666322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a062733c92a64fd68acb9a59fbba8b24', 'request_id': 'req-e5180876-984a-4619-a3c2-f639af6985a5'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.676888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] The token cache is being invalidate because project a062733c92a64fd68acb9a59fbba8b24 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:29:41.677176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.677784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:29:41.679234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1343/2668] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:29:41 2022] DELETE /identity/v3/projects/a062733c92a64fd68acb9a59fbba8b24 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:29:51.345955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=13,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:51.673430 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=20,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:29:58.030799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:29:58.032208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:58.088407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7474795-26e6-48f2-9f5e-e87e4ca5ed13 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:29:58.134836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:29:58.135508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:29:58.135791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:29:58.136066 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:29:58.137834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:29:58.201553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:29:58.202987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71631340-4122-49c5-8706-22a6acbb2686 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:29:58.213815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1326/2669] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:29:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:00.799594 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:00.801463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:00.816911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5180876-984a-4619-a3c2-f639af6985a5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:00.821536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:00.822185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:00.822467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:00.822740 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] PATH_INFO: `/v3/users/2e1287ffa55241d78592e098c09d4f83` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:00.824082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e1287ffa55241d78592e098c09d4f83)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:00.826914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:00.885545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e1287ffa55241d78592e098c09d4f83'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:00.901292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e1287ffa55241d78592e098c09d4f83'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:00.902201 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e1287ffa55241d78592e098c09d4f83'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:00.921975 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e1287ffa55241d78592e098c09d4f83'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:00.922696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e1287ffa55241d78592e098c09d4f83'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:00.957571 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1344/2670] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:00 2022] DELETE /identity/v3/users/2e1287ffa55241d78592e098c09d4f83 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:00.962967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:00.963455 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:00.963557 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:00.983983 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:00.985394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.056862 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1327/2671] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:01.066887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:01.068659 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.166483 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8a151a-454f-446b-b7f2-ae5261b06db2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:01.171315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:01.172286 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.172688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.174542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.186877 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:01.189375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:01.201890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1345/2672] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:01.451582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:01.453085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.476426 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48542c91-9207-4fce-bb6e-1d5d6c2eea74 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:01.484290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:01.484988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.485428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.485773 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] PATH_INFO: `/v3/users/53dab2ec3f2e4973893fb838c3d1d545` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.488733 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=53dab2ec3f2e4973893fb838c3d1d545)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:01.491262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:01.493359 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:01.554471 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '53dab2ec3f2e4973893fb838c3d1d545'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.577921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '53dab2ec3f2e4973893fb838c3d1d545'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.578716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '53dab2ec3f2e4973893fb838c3d1d545'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.580158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '53dab2ec3f2e4973893fb838c3d1d545'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.584071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:01.585748 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.589485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '53dab2ec3f2e4973893fb838c3d1d545'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.601172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaae9ba-8f5a-4052-be29-09b23123a8bc service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:01.607987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:01.609534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.609534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.609534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] PATH_INFO: `/v3/projects/c71eeab9c6314f68b5c7028be35e5134` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.611244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] RBAC: Authorizing `identity:delete_project(project_id=c71eeab9c6314f68b5c7028be35e5134)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:01.613105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1328/2673] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:01 2022] DELETE /identity/v3/users/53dab2ec3f2e4973893fb838c3d1d545 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:01.613783 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:01.690584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c71eeab9c6314f68b5c7028be35e5134', 'request_id': 'req-1e9eb982-0292-482f-ba56-a3324470eafc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.699723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c71eeab9c6314f68b5c7028be35e5134', 'request_id': 'req-1e9eb982-0292-482f-ba56-a3324470eafc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.707786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c71eeab9c6314f68b5c7028be35e5134', 'request_id': 'req-1e9eb982-0292-482f-ba56-a3324470eafc'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.717316 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:01.725628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] The token cache is being invalidate because project c71eeab9c6314f68b5c7028be35e5134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:01.726168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.727460 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.729371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1346/2674] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:01 2022] DELETE /identity/v3/projects/c71eeab9c6314f68b5c7028be35e5134 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:30:01.733204 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:01.734637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.798426 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:01.799905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:01.806182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fc878e-de15-4da5-af1a-5a7145acf1ec admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:01.868995 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9eb982-0292-482f-ba56-a3324470eafc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:01.869900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:01.870627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.870916 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.871298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] PATH_INFO: `/v3/users/a8d681b0607b4fc3a4f17875bd3a874b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.872598 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a8d681b0607b4fc3a4f17875bd3a874b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:01.873894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:01.874536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:01.874665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.874846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.875149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] PATH_INFO: `/v3/projects/ae629f9940d446f2ab39e9a49b3f4753` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.876746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae629f9940d446f2ab39e9a49b3f4753)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:01.878645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:01.906664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:01.929031 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae629f9940d446f2ab39e9a49b3f4753', 'request_id': 'req-de8c24d4-59bb-4fdb-8151-43caa866bbd8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.929222 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a8d681b0607b4fc3a4f17875bd3a874b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.936508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae629f9940d446f2ab39e9a49b3f4753', 'request_id': 'req-de8c24d4-59bb-4fdb-8151-43caa866bbd8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.942000 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a8d681b0607b4fc3a4f17875bd3a874b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.942694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a8d681b0607b4fc3a4f17875bd3a874b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.943306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a8d681b0607b4fc3a4f17875bd3a874b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.943601 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae629f9940d446f2ab39e9a49b3f4753', 'request_id': 'req-de8c24d4-59bb-4fdb-8151-43caa866bbd8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.951263 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-30471d2b-ca9d-4707-a524-a1e531396cb0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a8d681b0607b4fc3a4f17875bd3a874b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.955662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] The token cache is being invalidate because project ae629f9940d446f2ab39e9a49b3f4753 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:01.956008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.956670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-de8c24d4-59bb-4fdb-8151-43caa866bbd8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:01.958308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1347/2675] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:01 2022] DELETE /identity/v3/projects/ae629f9940d446f2ab39e9a49b3f4753 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:01.963219 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:01.963556 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:01.963836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:01.965262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1329/2676] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:01 2022] DELETE /identity/v3/users/a8d681b0607b4fc3a4f17875bd3a874b => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:01.986345 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:01.988297 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.083374 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1348/2677] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:02.092276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.092679 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.093119 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.113694 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:02.114966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.138840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1330/2678] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:02.140027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.152709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e2cd13-c0e7-4769-bb45-1d810331dada None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.157542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.158163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.158446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.158723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.159449 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:30:02.159768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.161512 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.173513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1349/2679] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:30:02.177686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.187459 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890da36f-f6fc-4c61-a4d7-dff7fdc3b773 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.193062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.193741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.194022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.194296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.194965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.196913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.226298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1331/2680] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:02.230536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.240428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f833a6-9dc1-40c8-bd35-1e1ad33dde95 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.255774 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:02.271627 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.273417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.314235 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.314886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.315165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.315463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.316130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.318057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.339559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1350/2681] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/users => generated 288 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:02.343584 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.345908 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2081c13c-2640-41ec-a0f8-d502b29588bd admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.350637 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.351277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.351599 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.351876 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] PATH_INFO: `/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.353048 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d57eb00bd0d64381b63ecf04475cb0f3)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.354930 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.357522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d55f13-1a0f-412b-a79b-4e26c58d1fcc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.365820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.366357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.366644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.366914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.367663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.369295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.383291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1351/2682] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:30:02.387881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.399115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8938de48-ad03-4cb9-a900-0e3d3274ff67 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.413647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd57eb00bd0d64381b63ecf04475cb0f3', 'request_id': 'req-85fea41a-286d-437d-8943-befb1c356ef7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:02.420010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd57eb00bd0d64381b63ecf04475cb0f3', 'request_id': 'req-85fea41a-286d-437d-8943-befb1c356ef7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:02.429368 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd57eb00bd0d64381b63ecf04475cb0f3', 'request_id': 'req-85fea41a-286d-437d-8943-befb1c356ef7'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:02.441490 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] The token cache is being invalidate because project d57eb00bd0d64381b63ecf04475cb0f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:02.441774 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:02.442417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-85fea41a-286d-437d-8943-befb1c356ef7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:02.443975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1332/2683] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:02 2022] DELETE /identity/v3/projects/d57eb00bd0d64381b63ecf04475cb0f3 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:30:02.448613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fcc7-cad4-4a1c-931b-bf2175350a54 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.448964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fcc7-cad4-4a1c-931b-bf2175350a54 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.449329 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e8fcc7-cad4-4a1c-931b-bf2175350a54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.455671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.456285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.456558 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.456827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.457506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.459075 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.463407 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:02.469230 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1352/2684] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:30:02.469608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-68e8fcc7-cad4-4a1c-931b-bf2175350a54 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:02.471481 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-68e8fcc7-cad4-4a1c-931b-bf2175350a54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.473481 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.475123 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.481093 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1333/2685] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:02.487888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfed42f-7739-4c51-be37-5a19193eadfc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.489312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.489604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.489880 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.492676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.493305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.493591 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.493865 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] PATH_INFO: `/v3/projects/1eec425dcc93489d9db53600cb95458d/users/1d69118a87374b14b0f85a23979d5264/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.509440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1eec425dcc93489d9db53600cb95458d, user_id=1d69118a87374b14b0f85a23979d5264, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.509658 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:02.510861 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.512110 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71b6eaad-f771-42ae-924a-aaa6969de00d admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.519811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1334/2686] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:02.523915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.525465 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1353/2687] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:02 2022] PUT /identity/v3/projects/1eec425dcc93489d9db53600cb95458d/users/1d69118a87374b14b0f85a23979d5264/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:30:02.530115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.530397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.530672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.532919 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ab9261-6e73-4cdd-a9ee-71611b61f4e3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.549291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:02.550924 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.589428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.590106 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.590333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.590613 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.591310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80121) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:30:02.591647 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.593228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.609735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1335/2688] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:30:02.609945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1354/2689] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:02.611174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.618432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.619797 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.621391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd77f36-545c-4509-930a-1c81a5c52b10 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.626669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.627281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.627601 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.627881 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.628583 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.629777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:02.630614 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.667662 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1336/2690] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/projects => generated 369 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:30:02.671781 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.678646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb59fc-d0a5-46ff-b4e8-07fbabd21e05 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.731147 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.731855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.732134 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.732412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.736438 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5959d776-7301-4d8c-a3d2-2bf5905aa829 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.741027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.741640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.741882 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.742004 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.742182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.742845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.743303 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.745725 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.753543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1355/2691] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:30:02.769576 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1337/2692] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/users => generated 285 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:30:02.773632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.784687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492a69a7-32f7-47df-ac65-ee9ecc3fada1 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.789553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.790165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.790444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.790720 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.791405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.793016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.803097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1356/2693] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:02.807209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.818839 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964c49f3-6694-4bf9-b191-95e560534ec6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.824085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.824707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.825332 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.825729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.826794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.828547 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24314959-10f2-4365-9454-1544c4c61472 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.841476 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1338/2694] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:02 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:30:02.846149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:02.856887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc8c5d3-6d62-43e5-b67b-4b890dfae024 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:02.861555 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:02.862218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.862498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.862775 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] PATH_INFO: `/v3/projects/5f767937f3e54c73a43800dfd8f9a4c4/users/9367bcfb486e480a990357fee6c27e05/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.879916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5f767937f3e54c73a43800dfd8f9a4c4, user_id=9367bcfb486e480a990357fee6c27e05, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:02.882563 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:02.898295 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1357/2695] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:02 2022] PUT /identity/v3/projects/5f767937f3e54c73a43800dfd8f9a4c4/users/9367bcfb486e480a990357fee6c27e05/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:02.909025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:02.909657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:02.910135 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:02.941844 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] MFA Rules not processed for user `9367bcfb486e480a990357fee6c27e05`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:02.943151 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:02.977825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:03.012971 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1339/2696] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:02 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:03.018084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.095062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647e6a26-fb77-490e-b9ab-aa60180d6e5e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.099926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.100625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.100979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.101323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] PATH_INFO: `/v3/projects` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.102179 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.104187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.138499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1358/2697] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:30:03 2022] POST /identity/v3/projects => generated 365 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:03.142766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.154086 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87220ceb-4b9f-43c0-9890-0a6c41875526 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.214375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.215298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.215612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.215887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.216634 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.218936 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.242603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1340/2698] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:30:03 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:30:03.246914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.258593 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb644a8-c39b-41a6-8377-59899326b50e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.263370 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.264033 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.264315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.264592 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.265309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.266925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.278978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1359/2699] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:30:03.291389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.304609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e826a6d-c395-46f9-b449-27e673a6cdb4 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.309501 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.310271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.310555 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.310834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] PATH_INFO: `/v3/projects/89ad8c39b24742abbbaaf1a9969868d6/users/295776312fba4f86a55aee7a9be34deb/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.327587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ad8c39b24742abbbaaf1a9969868d6, user_id=295776312fba4f86a55aee7a9be34deb, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.330309 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.344785 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1341/2700] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:03 2022] PUT /identity/v3/projects/89ad8c39b24742abbbaaf1a9969868d6/users/295776312fba4f86a55aee7a9be34deb/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:03.348940 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.355670 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:03.360751 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c147e98-61aa-419b-987b-e2343e160174 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.420152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.420778 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.421052 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.421318 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.422012 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.424285 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e34cd2c-c146-42e3-a1f5-a6313b80f54e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.434580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1360/2701] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:03 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:03.438712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.449028 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acb6e07-77f8-4809-a3eb-006be4d9c61d admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.454091 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.454850 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.455193 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.455574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] PATH_INFO: `/v3/projects/89ad8c39b24742abbbaaf1a9969868d6/users/295776312fba4f86a55aee7a9be34deb/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.457567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] RBAC: Authorizing `identity:create_grant(project_id=89ad8c39b24742abbbaaf1a9969868d6, user_id=295776312fba4f86a55aee7a9be34deb, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.460291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.471617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1342/2702] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:03 2022] PUT /identity/v3/projects/89ad8c39b24742abbbaaf1a9969868d6/users/295776312fba4f86a55aee7a9be34deb/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:03.482128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.482368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.482642 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.511211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] MFA Rules not processed for user `295776312fba4f86a55aee7a9be34deb`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:03.512484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:03.583412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1361/2703] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:03 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:03.609217 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.612156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:03.686399 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84cdad17-c634-4e08-8ee1-79c8c4d81616 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.690349 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.691154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.691565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.691932 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.702272 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.704185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf29669-7ec7-4dae-a629-e2d7fac65d12 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.715435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1343/2704] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:03 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:03.773606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:03.775212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:03.791912 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c017404b-3dd7-4d60-af4f-cc191e301ae9 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:03.795956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:03.796822 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:03.797263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:03.797738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:03.859006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:03.860598 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:03.870091 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1362/2705] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:03 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:06.849684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:06.850179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:06.850287 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:06.885944 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] MFA Rules not processed for user `1d69118a87374b14b0f85a23979d5264`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:06.887259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:06.910559 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:06.990265 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1344/2706] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:07.005380 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:07.007486 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:07.040926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=100 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:07.094032 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bc362e-c563-4cb3-a740-c0d479316347 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:07.100523 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:07.101293 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:07.101636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:07.101978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:07.111614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:07.113314 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df0fbea1-cc0e-49b6-992c-edabe831e8d5 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:07.123455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1363/2707] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:07.631890 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:07.737806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9171c2c6-f0f1-4047-b3dd-521d51c71ffa None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:07.747537 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:07.748298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:07.748642 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:07.749044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] PATH_INFO: `/v3/users/9367bcfb486e480a990357fee6c27e05` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:07.750411 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9367bcfb486e480a990357fee6c27e05)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:07.752574 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:07.821367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9367bcfb486e480a990357fee6c27e05'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:07.837745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9367bcfb486e480a990357fee6c27e05'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:07.838498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9367bcfb486e480a990357fee6c27e05'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:07.839150 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9367bcfb486e480a990357fee6c27e05'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:07.851326 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9367bcfb486e480a990357fee6c27e05'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:07.872942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1345/2708] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:07 2022] DELETE /identity/v3/users/9367bcfb486e480a990357fee6c27e05 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:07.877490 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:07.878016 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:07.878470 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:07.902281 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:07.903964 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:07.983862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1364/2709] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:07.995277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:07.997040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:08.062017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:08.085141 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a528d7cc-1cdd-4d54-8f33-4e5141f8d9c9 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:08.089437 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:08.090390 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:08.090720 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:08.091070 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:08.105621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:08.108956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:08.124386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1346/2710] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:08.478626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:08.480178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:08.497343 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ee6ff9a-c54b-4b39-8d3b-1950ed9c89d6 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:08.502501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:08.503405 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:08.503889 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:08.504169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] PATH_INFO: `/v3/projects/5f767937f3e54c73a43800dfd8f9a4c4` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:08.510187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] RBAC: Authorizing `identity:delete_project(project_id=5f767937f3e54c73a43800dfd8f9a4c4)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:08.513672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:08.569331 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5f767937f3e54c73a43800dfd8f9a4c4', 'request_id': 'req-a7ace6f7-77f7-45a3-be11-a7673016d00f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.580212 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5f767937f3e54c73a43800dfd8f9a4c4', 'request_id': 'req-a7ace6f7-77f7-45a3-be11-a7673016d00f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.588987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5f767937f3e54c73a43800dfd8f9a4c4', 'request_id': 'req-a7ace6f7-77f7-45a3-be11-a7673016d00f'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.600153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] The token cache is being invalidate because project 5f767937f3e54c73a43800dfd8f9a4c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:08.600447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.601100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7ace6f7-77f7-45a3-be11-a7673016d00f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.602687 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1365/2711] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:08 2022] DELETE /identity/v3/projects/5f767937f3e54c73a43800dfd8f9a4c4 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:08.606653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:08.616877 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e61d0a-45fb-4fe8-a86b-82a99dcdf3eb service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:08.675523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:08.676283 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:08.676640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:08.677452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] PATH_INFO: `/v3/users/295776312fba4f86a55aee7a9be34deb` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:08.678228 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=295776312fba4f86a55aee7a9be34deb)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:08.680354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:08.725890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077d59-3a28-4756-93b7-31573f655921 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:08.726177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077d59-3a28-4756-93b7-31573f655921 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:08.726455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a077d59-3a28-4756-93b7-31573f655921 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:08.733929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '295776312fba4f86a55aee7a9be34deb'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.745288 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4a077d59-3a28-4756-93b7-31573f655921 None None] MFA Rules not processed for user `1d69118a87374b14b0f85a23979d5264`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:08.746034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '295776312fba4f86a55aee7a9be34deb'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.746733 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4a077d59-3a28-4756-93b7-31573f655921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:08.746863 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '295776312fba4f86a55aee7a9be34deb'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.747400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '295776312fba4f86a55aee7a9be34deb'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.756227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '295776312fba4f86a55aee7a9be34deb'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:08.770480 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1347/2712] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:08 2022] DELETE /identity/v3/users/295776312fba4f86a55aee7a9be34deb => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:08.783768 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:08.813663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1366/2713] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:09.089428 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.090917 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:09.186959 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2268d20-40c3-415b-93ec-7b9c233d24e2 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.193267 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.193938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.194221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.194498 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] PATH_INFO: `/v3/projects/89ad8c39b24742abbbaaf1a9969868d6` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.195800 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=89ad8c39b24742abbbaaf1a9969868d6)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.197718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.199819 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:09.251192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '89ad8c39b24742abbbaaf1a9969868d6', 'request_id': 'req-457f8d5d-fe96-4475-b8b7-6afe8351a16b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:09.257436 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '89ad8c39b24742abbbaaf1a9969868d6', 'request_id': 'req-457f8d5d-fe96-4475-b8b7-6afe8351a16b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:09.264746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '89ad8c39b24742abbbaaf1a9969868d6', 'request_id': 'req-457f8d5d-fe96-4475-b8b7-6afe8351a16b'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:09.276161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] The token cache is being invalidate because project 89ad8c39b24742abbbaaf1a9969868d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:09.276540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:09.277243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-457f8d5d-fe96-4475-b8b7-6afe8351a16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:09.278883 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1348/2714] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:09 2022] DELETE /identity/v3/projects/89ad8c39b24742abbbaaf1a9969868d6 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:09.285202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.285701 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.285985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.308862 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:09.310119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:09.383139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1367/2715] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:30:09.452473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.452760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.453032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.475335 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:09.476587 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:09.485921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1349/2716] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:09.490145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.502694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d7bed0-8bc0-478f-a9d2-ef3bb3aefc32 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.524861 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.525984 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.526770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.527322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.529066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:30:09.529550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.531309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.554736 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1368/2717] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:30:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:30:09.561072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.571372 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7035a9d-e9e9-4281-aea8-9af15903f9e4 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.576957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.578321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.578791 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.579226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.580388 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.583617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.627541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1350/2718] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:30:09 2022] POST /identity/v3/projects => generated 375 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:09.630008 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.646950 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4336a81a-9279-4d17-8be0-bd0e22144377 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.716729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.717290 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.717640 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.717982 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.718771 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.721305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.745697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1369/2719] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:30:09 2022] POST /identity/v3/users => generated 288 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:09.750125 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.765699 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e840be-e407-4c51-97a0-e2e026407cb0 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.770935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.771512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.771794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.772069 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.772772 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.774433 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.788701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1351/2720] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:09.792566 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.804645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5add31d-7137-46d9-80a3-c218003aa410 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.810629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.811666 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.812042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.812404 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.813806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.815714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b05b01cd-89ca-4f2a-9003-d73be74a0f64 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.829335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1370/2721] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:09 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:09.834138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:09.847143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c277e7af-7baa-450f-8656-787eac41b5da admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:09.853512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:09.854382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.854734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.855241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] PATH_INFO: `/v3/projects/cecd6fc0c634487383617c8d323a185b/users/a1677d32153947d5bdbd24642f3890a1/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.875102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] RBAC: Authorizing `identity:create_grant(project_id=cecd6fc0c634487383617c8d323a185b, user_id=a1677d32153947d5bdbd24642f3890a1, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:09.878946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:09.898050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1352/2722] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:09 2022] PUT /identity/v3/projects/cecd6fc0c634487383617c8d323a185b/users/a1677d32153947d5bdbd24642f3890a1/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:09.910506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:09.911001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:09.911381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:09.950749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] MFA Rules not processed for user `a1677d32153947d5bdbd24642f3890a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:09.952333 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:09.992205 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:10.043688 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1371/2723] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:09 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:10.056892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:10.059189 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:10.123190 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb62d4f0-5ca4-4226-82bc-b763f310886b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:10.135767 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:10.136695 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:10.136809 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:10.137084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:10.158242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:10.159794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a4b748-efa7-4029-9e7d-424f3870c2d5 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:10.170405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1353/2724] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:10 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:20.039040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=12 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:20.165236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=77,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:27.939260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.014043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fd7565-463f-4f0d-b654-5081ba2aba29 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:28.019313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:28.020001 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.020308 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.020578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] PATH_INFO: `/v3/users/a1677d32153947d5bdbd24642f3890a1` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.021753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=a1677d32153947d5bdbd24642f3890a1)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:28.023709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:28.075211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a1677d32153947d5bdbd24642f3890a1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.097317 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a1677d32153947d5bdbd24642f3890a1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.098102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a1677d32153947d5bdbd24642f3890a1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.107347 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a1677d32153947d5bdbd24642f3890a1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.108239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a1677d32153947d5bdbd24642f3890a1'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.121830 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1372/2725] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:30:27 2022] DELETE /identity/v3/users/a1677d32153947d5bdbd24642f3890a1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:28.127246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.127754 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.128007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.151115 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:28.152403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:28.226745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1354/2726] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:28.240630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.243530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:28.305402 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ce080-47e4-4d97-a6b8-465e2391fb4f admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:28.309190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:28.309818 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.310102 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.310379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.320419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:28.321860 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e279404-f170-48b6-9c96-e558a27b22a1 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:28.332176 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1373/2727] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:28.627621 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.628912 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:28.644974 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321e0aa4-ca02-4b45-9830-df205d0a3297 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:28.651450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:28.652037 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.652314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.652590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] PATH_INFO: `/v3/projects/cecd6fc0c634487383617c8d323a185b` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.653842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cecd6fc0c634487383617c8d323a185b)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:28.655796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:28.708512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cecd6fc0c634487383617c8d323a185b', 'request_id': 'req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.715008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cecd6fc0c634487383617c8d323a185b', 'request_id': 'req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.723666 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cecd6fc0c634487383617c8d323a185b', 'request_id': 'req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.736677 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] The token cache is being invalidate because project cecd6fc0c634487383617c8d323a185b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:30:28.736966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.737789 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-3c60d3b2-c9bd-4f3e-a467-3186a655fc48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:30:28.739709 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1355/2728] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:30:28 2022] DELETE /identity/v3/projects/cecd6fc0c634487383617c8d323a185b => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:28.744541 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.744825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.745100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.764915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:28.766607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:28.784686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:28.831151 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1374/2729] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:28.838941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.839232 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.839569 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.860247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:28.861520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:28.869899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1356/2730] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:28.874362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.884498 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559b57ef-4bd7-4d15-a43d-64474f821fac None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:28.889263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:28.889876 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.890185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.890980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.891711 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:30:28.892015 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:28.893605 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:28.905854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1375/2731] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:30:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:30:28.911219 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.921223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e08778f-22f7-478d-ad47-5914d400406e None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:28.927226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:28.927926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:28.928208 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:28.928483 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:28.929168 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:28.931065 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:28.960688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1357/2732] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:30:28 2022] POST /identity/v3/projects => generated 363 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:28.964972 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:28.975357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc50fb6-f8ee-4465-bf37-b9798ee13ebb admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:29.033261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:29.033902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.034162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.034440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.035148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:29.037135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:29.058810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1376/2733] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:30:28 2022] POST /identity/v3/users => generated 282 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:30:29.063646 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:29.074511 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5daa3802-c3d8-4bbe-a494-21cfaed1326b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:29.080333 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:29.080979 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.081258 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.081529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.082226 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:29.083865 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:29.094447 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1358/2734] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:29.098729 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:29.109455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea7ab23-5baf-480b-a9eb-fb6904e840ba admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:29.114823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:29.115497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.115989 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.116275 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.117040 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:29.118752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab542a70-7a19-48d4-8137-c21d339505d3 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:29.131177 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1377/2735] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:30:29 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:30:29.135536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:29.152196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddfd66-d809-4a9b-92b5-ff88628003b7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:29.157276 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:29.158042 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.158391 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.158739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] PATH_INFO: `/v3/projects/f1e4c8f91aba4cb59c883e26f8da48a1/users/43d5d5163a364439a4f56c1995d3a1c0/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.177243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1e4c8f91aba4cb59c883e26f8da48a1, user_id=43d5d5163a364439a4f56c1995d3a1c0, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:29.180114 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:29.193967 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1359/2736] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:30:29 2022] PUT /identity/v3/projects/f1e4c8f91aba4cb59c883e26f8da48a1/users/43d5d5163a364439a4f56c1995d3a1c0/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:30:29.206211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.206662 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.207017 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.235238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] MFA Rules not processed for user `43d5d5163a364439a4f56c1995d3a1c0`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:30:29.236452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:29.304980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1378/2737] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:30:29 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:30:29.316673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:30:29.318444 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:30:29.342556 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:29.382886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016933e4-3363-44f8-8874-9b2e7e0677bf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:30:29.386735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:30:29.387366 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:30:29.387766 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:30:29.387975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:30:29.397200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:30:29.398643 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f76a861-072f-4e2e-889e-44dd0acff817 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:30:29.407727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1360/2738] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:30:29 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:30:39.301848 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:30:39.403529 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=17 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:04.642599 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:04.749457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4c02a-ba14-4a5c-b7ac-cc07afe0ae63 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:04.754660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:04.755298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:04.755617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:04.755893 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] PATH_INFO: `/v3/users/43d5d5163a364439a4f56c1995d3a1c0` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:04.757057 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43d5d5163a364439a4f56c1995d3a1c0)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:04.758953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:04.803791 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43d5d5163a364439a4f56c1995d3a1c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:04.816024 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43d5d5163a364439a4f56c1995d3a1c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:04.817018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43d5d5163a364439a4f56c1995d3a1c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:04.825055 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43d5d5163a364439a4f56c1995d3a1c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:04.825716 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43d5d5163a364439a4f56c1995d3a1c0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:04.837887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1379/2739] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:31:04 2022] DELETE /identity/v3/users/43d5d5163a364439a4f56c1995d3a1c0 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:31:04.842338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:04.842806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:04.842921 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:04.892836 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:04.894027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:04.951759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1361/2740] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:04.961753 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:04.963150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:05.004746 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:05.058945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5034b9f-5f53-43ec-8e25-d150174ff829 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.062570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.063182 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.063499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.063803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.072649 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.074074 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6178bf06-4f20-4eb6-afef-3f39b43997d4 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.083700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1380/2741] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:31:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:31:05.349552 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.350846 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:05.363218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d629ad75-3cd5-48ec-a502-db7b73a41c69 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.367869 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.368524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.368806 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.369081 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] PATH_INFO: `/v3/projects/f1e4c8f91aba4cb59c883e26f8da48a1` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.370245 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1e4c8f91aba4cb59c883e26f8da48a1)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.372202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.418284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1e4c8f91aba4cb59c883e26f8da48a1', 'request_id': 'req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:05.424384 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1e4c8f91aba4cb59c883e26f8da48a1', 'request_id': 'req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:05.431724 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1e4c8f91aba4cb59c883e26f8da48a1', 'request_id': 'req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:05.442239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] The token cache is being invalidate because project f1e4c8f91aba4cb59c883e26f8da48a1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:31:05.442523 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:05.443161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-a3bf59ba-16f7-41f9-a26a-2f4edf1e5130 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:31:05.444707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1362/2742] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:31:05 2022] DELETE /identity/v3/projects/f1e4c8f91aba4cb59c883e26f8da48a1 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:31:05.449632 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.450171 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.450446 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.498163 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:05.499349 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:05.555851 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1381/2743] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:05.563797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.564083 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.564367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.584669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:05.585867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:05.593824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1363/2744] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:05.598673 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.608269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742be82d-6b49-42a5-a732-5bba7c25d85b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.612897 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.613508 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.613787 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.614135 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.614739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:31:05.615070 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.616744 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.628707 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1382/2745] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:31:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:31:05.633203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.643277 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf9f0c2-f988-4eb5-96e0-63efbfd2ccce None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.648269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.648892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.649199 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.649478 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.650148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.652130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.680184 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1364/2746] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:31:05 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:31:05.684553 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.695359 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e52d72a-7422-4e8b-9cbb-f5139e1062b2 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.756257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.756966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.757239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.757617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.758694 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.761315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.780253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:05.791013 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1383/2747] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:31:05 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:31:05.792238 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.802955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac3a7cf-d74f-4ac5-9c48-39e376392332 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.808845 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.809593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.809875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.810154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.810860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.812759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.826253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1365/2748] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:31:05.832049 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.842120 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f026a1-aee9-4a0a-a812-5ebe275959ab admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.847489 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.848148 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.848432 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.848710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.849415 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.851257 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7344f31e-9cc9-49b2-86bd-123f5f7d7359 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.862974 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1384/2749] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:31:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:31:05.867247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:05.879206 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba9a9754-891e-4005-b049-d7a628ba45bf admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:05.883624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:05.884302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.884585 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.884926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] PATH_INFO: `/v3/projects/8561e7f1f3d040e28d5084ebf937e198/users/0bbe35ad557841c581e3b1a4aa272f38/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.912256 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8561e7f1f3d040e28d5084ebf937e198, user_id=0bbe35ad557841c581e3b1a4aa272f38, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:05.915128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:05.917871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:05.932084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1366/2750] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:31:05 2022] PUT /identity/v3/projects/8561e7f1f3d040e28d5084ebf937e198/users/0bbe35ad557841c581e3b1a4aa272f38/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:31:05.937210 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2470-ccc4-4993-bde5-f0bf881bdcc8 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:05.937497 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2470-ccc4-4993-bde5-f0bf881bdcc8 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:05.937793 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c2470-ccc4-4993-bde5-f0bf881bdcc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:05.958410 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-8a6c2470-ccc4-4993-bde5-f0bf881bdcc8 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:05.959654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8a6c2470-ccc4-4993-bde5-f0bf881bdcc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:06.018803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1385/2751] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:06.028379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:06.030134 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:06.088650 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc21105-c51b-4fcc-8bdb-a89e3ace2926 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:06.092308 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:06.092943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:06.093224 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:06.093513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:06.102432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:06.103896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:06.113608 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1367/2752] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:31:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:31:10.277412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.277870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.278088 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.309382 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] MFA Rules not processed for user `0bbe35ad557841c581e3b1a4aa272f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:10.310872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:10.412738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1386/2753] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:10.417255 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.428124 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800de5df-5b68-43f0-ada6-e66784030082 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.432714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.433331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.433604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.433874 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.434546 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.436507 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.471952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1368/2754] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:31:10 2022] POST /identity/v3/projects => generated 377 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:31:10.476085 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.487931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd28ff3-bf8f-482a-be1a-62f5e3ae4f9c None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.547099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.547807 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.548092 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.548369 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.549061 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.551019 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.573693 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1387/2755] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:31:10 2022] POST /identity/v3/users => generated 295 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:31:10.577903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.589223 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c52322-90cb-4d5f-bbc2-0ac58a365d05 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.593886 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.594514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.595343 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.595657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.596367 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.597988 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.609485 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1369/2756] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:31:10.613706 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.624062 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8a068a-e2d2-4a6b-9ec7-41dcf211025a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.630397 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.631185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.631613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.631963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] PATH_INFO: `/v3/projects/7e920c5683554b9a830c8d4ec011059f/users/c499de2c372642ff91e409f8c2882a74/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.635368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:10.651174 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e920c5683554b9a830c8d4ec011059f, user_id=c499de2c372642ff91e409f8c2882a74, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.653934 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.668071 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1388/2757] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:31:10 2022] PUT /identity/v3/projects/7e920c5683554b9a830c8d4ec011059f/users/c499de2c372642ff91e409f8c2882a74/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:31:10.672167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.682796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9713f8-500a-4069-ab3c-9e5262eb8bc6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.742898 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.743484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.743765 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.744061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.744769 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.746493 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfc5851-3d4f-4cfc-ab23-0c18d66736e1 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.769515 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1370/2758] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:31:10 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:31:10.770843 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:10.785835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff46044-c219-4fa8-9c8e-6c1a3e4cdb02 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:10.790495 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:10.791165 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:10.791476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:10.791754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] PATH_INFO: `/v3/projects/7e920c5683554b9a830c8d4ec011059f/users/c499de2c372642ff91e409f8c2882a74/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:10.793695 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e920c5683554b9a830c8d4ec011059f, user_id=c499de2c372642ff91e409f8c2882a74, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:10.798475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:10.813223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1389/2759] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:31:10 2022] PUT /identity/v3/projects/7e920c5683554b9a830c8d4ec011059f/users/c499de2c372642ff91e409f8c2882a74/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:31:15.123133 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d52c-151e-4448-b2d3-2a2203b7d6b6 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:15.124036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d52c-151e-4448-b2d3-2a2203b7d6b6 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:15.124348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee90d52c-151e-4448-b2d3-2a2203b7d6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:15.156756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ee90d52c-151e-4448-b2d3-2a2203b7d6b6 None None] MFA Rules not processed for user `c499de2c372642ff91e409f8c2882a74`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:15.158616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ee90d52c-151e-4448-b2d3-2a2203b7d6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:15.211900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:15.236050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1371/2760] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:15 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:15.250029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:15.252994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:15.392276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f70be8-03f7-4560-b04d-ee129263d8cc admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:15.400269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:15.400993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:15.401348 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:15.401727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:15.464531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:15.466145 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:15.476610 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1390/2761] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:31:15 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:31:25.232729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=7 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:25.472276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=80,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:30.787285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:30.787818 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:30.787941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:30.806882 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] MFA Rules not processed for user `0bbe35ad557841c581e3b1a4aa272f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:31:30.808103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:30.818477 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1372/2762] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:31:30 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:31:32.027284 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:32.028761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:32.118510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882fc577-5e4a-4592-a6b8-8f983757f2a8 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:32.123607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:32.124234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:32.124668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:32.124945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:32.126755 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:32.138310 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:32.139918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:32.149963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1391/2763] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:31:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:31:40.802306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:42.145806 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=41 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:31:46.938068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:31:46.939500 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:31:46.997682 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561a8782-265c-4875-bbab-2b6a0d7718df None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:31:47.001446 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:31:47.002071 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:31:47.002400 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:31:47.002673 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:31:47.011616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:31:47.013034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:31:47.022257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1373/2764] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:31:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:31:57.018075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=26 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:32:49.682534 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:32:49.691904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c3dd538-7a90-473f-828f-5e9eb985fb50 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:32:49.695545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:32:49.696153 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:32:49.696434 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:32:49.696710 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:32:49.698452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:32:49.712187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:32:49.713606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb1b316-7d06-4ae6-96f3-7c12fa2b0e6c service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:32:49.724886 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1392/2765] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:32:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:32:50.905424 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:32:50.914076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e41918-b5a5-4909-ac78-d3b570ac52af service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:32:50.917592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:32:50.918194 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:32:50.918473 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:32:50.918745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:32:50.920486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:32:50.933920 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:32:50.935364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:32:50.946708 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1374/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:32:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:32:56.781178 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:32:56.781798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:32:56.781798 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:32:56.801208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] MFA Rules not processed for user `1d69118a87374b14b0f85a23979d5264`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:32:56.802409 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:32:56.855260 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1393/2767] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:32:56 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:32:57.152595 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:32:57.161687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d1f56b-347a-4d49-9291-e84c6cca2b90 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:32:57.165860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:32:57.166474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:32:57.166753 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:32:57.167026 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:32:57.168788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:32:57.179375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:32:57.180815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:32:57.188687 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1375/2768] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:32:57 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:33:06.848899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=31 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:33:07.185104 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=12 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:33:08.328542 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:33:08.338428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0661e6f-bd58-44a0-a17a-acc93f31e361 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:33:08.342158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:33:08.342777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:33:08.343054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:33:08.343322 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:33:08.345109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:33:08.354597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:33:08.356050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf154341-2240-4750-849c-7dcaf9927e1f service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:33:08.364447 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1394/2769] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:33:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:33:18.360519 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:33:48.218701 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:33:48.232899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fd8e420-5076-483f-af99-c4dcc9b70d09 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:33:48.248273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:33:48.249033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:33:48.249386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:33:48.249681 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:33:48.251743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:33:48.265825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:33:48.268205 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:33:48.283759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1376/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:33:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:33:49.226721 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:33:49.227239 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:33:49.227338 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:33:49.246546 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] MFA Rules not processed for user `0bbe35ad557841c581e3b1a4aa272f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:33:49.247784 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:33:49.255737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1395/2771] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:33:49 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:33:50.003075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:33:50.004540 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:33:50.018698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9700dcfb-b42c-4151-9d35-584c5993c484 service nova] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:33:50.023648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:33:50.024273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:33:50.024554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:33:50.024829 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:33:50.026590 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:33:50.039715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:33:50.041296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:33:50.055484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1377/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:33:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:33:59.241385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:00.050430 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=12 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:03.188795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:03.190887 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:03.209095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb92138-fbf5-4dff-a19c-9006dba95910 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:03.214878 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:03.215533 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:03.215819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:03.216095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:03.218168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:03.232261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:03.233852 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:03.244568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1396/2773] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:34:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:34:13.239980 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:18.808113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:18.822577 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a869504-775b-4922-8b52-3601d9c93c9f service placement] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:18.829262 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:18.830241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:18.830571 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:18.830956 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:18.834340 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:18.847736 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:18.849712 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:18.861792 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1378/2774] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:34:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:34:18.872958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:18.874271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:18.949325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f2d845-225e-4108-860c-49f92e58ff36 service nova] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:18.954559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:18.955202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:18.955515 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:18.955794 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] PATH_INFO: `/v3/users/1d69118a87374b14b0f85a23979d5264` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:18.956960 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d69118a87374b14b0f85a23979d5264)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:18.958907 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.005412 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d69118a87374b14b0f85a23979d5264'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.016803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d69118a87374b14b0f85a23979d5264'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.017452 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d69118a87374b14b0f85a23979d5264'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.024570 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d69118a87374b14b0f85a23979d5264'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.025202 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-a7921548-8f6c-4437-9b02-9a1649a2d661 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d69118a87374b14b0f85a23979d5264'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.041043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1397/2775] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:34:18 2022] DELETE /identity/v3/users/1d69118a87374b14b0f85a23979d5264 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:19.221807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.229280 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41968706-c3ec-4ead-bf73-a4436c910cbb service placement] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.291361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.291961 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.292243 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.292519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] PATH_INFO: `/v3/projects/1eec425dcc93489d9db53600cb95458d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.293669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1eec425dcc93489d9db53600cb95458d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.295894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.340952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1eec425dcc93489d9db53600cb95458d', 'request_id': 'req-5060b101-1cd9-4517-99e5-ad48abd0dfe2'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.346935 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1eec425dcc93489d9db53600cb95458d', 'request_id': 'req-5060b101-1cd9-4517-99e5-ad48abd0dfe2'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.354674 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1eec425dcc93489d9db53600cb95458d', 'request_id': 'req-5060b101-1cd9-4517-99e5-ad48abd0dfe2'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.365314 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] The token cache is being invalidate because project 1eec425dcc93489d9db53600cb95458d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:34:19.365597 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.366207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-5060b101-1cd9-4517-99e5-ad48abd0dfe2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:19.367716 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1379/2776] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:34:19 2022] DELETE /identity/v3/projects/1eec425dcc93489d9db53600cb95458d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:19.372678 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.372966 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.373240 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.393185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:34:19.394391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:19.451118 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1398/2777] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:34:19.459827 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.460110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.460382 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.479468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:34:19.480648 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:19.488305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1380/2778] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:34:19.492469 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.501846 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f2e37c-9d31-4266-8418-e977ca0a7c39 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.506842 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.507468 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.507749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.508018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.508705 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:34:19.509005 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.510586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.522476 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1399/2779] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:34:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 31 14:34:19.527596 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.536722 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09feae7-5896-4476-8b68-649ee11c4080 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.541291 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.541963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.542166 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.542440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.543102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.545054 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.580965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1381/2780] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:34:19.585786 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.595749 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c91eee-5bf5-482e-843c-a03e9d47830b admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.599244 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:19.653128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.653732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.654009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.654282 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.654923 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.656858 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.680175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1400/2781] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:34:19.681926 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.691941 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83855e9-7b65-460a-bd55-943bc453cd9f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.697416 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.698062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.698346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.698623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.699327 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.700948 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.711111 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1382/2782] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:34:19.714968 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.724890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07d0915-d350-4147-9238-36644c303e98 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.729421 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.730014 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.730292 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.730583 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.731253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.732847 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e614cc-538a-436e-9e28-2f65dfcd2c84 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.743606 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1401/2783] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:34:19 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:34:19.747553 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.757885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de724770-9885-487e-bb2e-af5a8f7f36ce admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.762371 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.763018 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.763293 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.763604 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] PATH_INFO: `/v3/projects/259965fbf4ed437c8565f6f1f4107641/users/33d34c8b8c1a4305bd24caafb0e8c940/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.778033 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] RBAC: Authorizing `identity:create_grant(project_id=259965fbf4ed437c8565f6f1f4107641, user_id=33d34c8b8c1a4305bd24caafb0e8c940, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.780745 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:19.793962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1383/2784] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:34:19 2022] PUT /identity/v3/projects/259965fbf4ed437c8565f6f1f4107641/users/33d34c8b8c1a4305bd24caafb0e8c940/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:34:19.805963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.806252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.806532 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.834192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] MFA Rules not processed for user `33d34c8b8c1a4305bd24caafb0e8c940`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:34:19.835437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:19.898672 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1402/2785] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:34:19.903179 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:19.908739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:19.966538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8583a16-7acf-4d45-bbf5-a0227602ecce admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:19.971156 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:19.971798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:19.972082 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:19.972357 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:19.973017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:19.975162 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.001567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1384/2786] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:34:19 2022] POST /identity/v3/projects => generated 373 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:34:20.006340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.016863 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1bbca9-f92c-49d8-a2eb-2ce829f2e50f None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.072918 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.073474 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.073742 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.074023 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.074684 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.076898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.097423 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1403/2787] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:34:20 2022] POST /identity/v3/users => generated 293 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:34:20.101946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.112306 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb7638e4-74b9-4884-811f-2bf0bf74c75b admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.117757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.118377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.118657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.118929 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.119659 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.121264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.135835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1385/2788] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:34:20.137361 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.144247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:20.149129 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b2189e-ddba-4ae0-b919-c1857b54aee0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.155223 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.156036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.156325 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.156607 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] PATH_INFO: `/v3/projects/4e617d028c6d4448bc751413bf3c459d/users/d4c6adbd09e8489b8349343477e85802/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.172770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e617d028c6d4448bc751413bf3c459d, user_id=d4c6adbd09e8489b8349343477e85802, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.175501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.189911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1404/2789] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:34:20 2022] PUT /identity/v3/projects/4e617d028c6d4448bc751413bf3c459d/users/d4c6adbd09e8489b8349343477e85802/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:20.194046 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.204072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c93a9f3-8cf7-45e7-8092-d4f6378bcd5f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.259888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.260497 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.260771 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.261050 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.261721 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.263286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04b61a94-b21d-4359-88d5-fa502ae42d79 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.272943 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1386/2790] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:34:20.277175 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.287819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d29f75-f657-4a84-8044-fc82c722e5a0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.292471 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.293107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.293388 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.293664 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] PATH_INFO: `/v3/projects/4e617d028c6d4448bc751413bf3c459d/users/d4c6adbd09e8489b8349343477e85802/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.295644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e617d028c6d4448bc751413bf3c459d, user_id=d4c6adbd09e8489b8349343477e85802, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.298271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.308994 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1405/2791] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:34:20 2022] PUT /identity/v3/projects/4e617d028c6d4448bc751413bf3c459d/users/d4c6adbd09e8489b8349343477e85802/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:20.319096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.319387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.319812 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.347369 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] MFA Rules not processed for user `d4c6adbd09e8489b8349343477e85802`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:34:20.349045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:20.412496 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:20.417157 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1387/2792] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:34:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 31 14:34:20.428198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.429937 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:20.507277 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a104042-13b1-4af1-82f5-e8fe3514ec8e admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.510985 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.511631 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.511911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.512187 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.521702 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.523127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.532524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1406/2793] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:34:20.606746 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:20.616524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f24857cb-41d8-49bf-bc34-f84036567d93 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:20.620079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:20.620689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:20.620969 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:20.621242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:20.678259 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:20.679718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4780b19d-592b-4843-92aa-97181186c053 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:20.688514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1388/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:34:20 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:34:30.528328 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=48,keystone:INSERT=2 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:30.684389 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=28 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:58.970804 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:59.030708 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b2643c-9148-4f46-95ad-2d83c18b0394 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:59.035149 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:59.035809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:59.036086 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:59.036358 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] PATH_INFO: `/v3/users/33d34c8b8c1a4305bd24caafb0e8c940` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:59.037494 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33d34c8b8c1a4305bd24caafb0e8c940)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:59.039389 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:59.086323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33d34c8b8c1a4305bd24caafb0e8c940'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.098009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33d34c8b8c1a4305bd24caafb0e8c940'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.098663 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33d34c8b8c1a4305bd24caafb0e8c940'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.105727 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33d34c8b8c1a4305bd24caafb0e8c940'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.106340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33d34c8b8c1a4305bd24caafb0e8c940'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.118747 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1407/2795] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:34:58 2022] DELETE /identity/v3/users/33d34c8b8c1a4305bd24caafb0e8c940 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:59.123203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:59.123742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:59.123841 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:59.142665 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:34:59.143904 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:59.202288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1389/2796] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:34:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:34:59.211855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:59.213245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:59.268763 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f52b6f-b382-4dcc-a85f-90ae577ba411 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:59.272518 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:59.273130 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:59.273411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:59.273686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:59.283400 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:59.284871 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:59.294455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1408/2797] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:34:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:34:59.560034 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:59.561522 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:34:59.573670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c858e6a-a2d8-46ce-9431-f5aadb201f3a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:59.579375 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:59.580067 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:59.580345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:59.580738 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] PATH_INFO: `/v3/projects/259965fbf4ed437c8565f6f1f4107641` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:59.581907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] RBAC: Authorizing `identity:delete_project(project_id=259965fbf4ed437c8565f6f1f4107641)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:59.583858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:59.630973 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '259965fbf4ed437c8565f6f1f4107641', 'request_id': 'req-4e63f825-cdf7-4908-a743-9001faac3b41'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.637567 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '259965fbf4ed437c8565f6f1f4107641', 'request_id': 'req-4e63f825-cdf7-4908-a743-9001faac3b41'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.645788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '259965fbf4ed437c8565f6f1f4107641', 'request_id': 'req-4e63f825-cdf7-4908-a743-9001faac3b41'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.659691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] The token cache is being invalidate because project 259965fbf4ed437c8565f6f1f4107641 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:34:59.659975 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.660606 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.662107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1390/2798] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:34:59 2022] DELETE /identity/v3/projects/259965fbf4ed437c8565f6f1f4107641 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:34:59.666324 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:34:59.676246 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf68cf5-767b-4264-b555-46de8b621fa0 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:34:59.694796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:34:59.733803 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:34:59.734440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:34:59.734719 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:34:59.735021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] PATH_INFO: `/v3/users/d4c6adbd09e8489b8349343477e85802` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:34:59.736217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d4c6adbd09e8489b8349343477e85802)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:34:59.738127 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:34:59.783773 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd4c6adbd09e8489b8349343477e85802'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.795350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd4c6adbd09e8489b8349343477e85802'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.796039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd4c6adbd09e8489b8349343477e85802'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.803080 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd4c6adbd09e8489b8349343477e85802'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.804413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-3a57d22d-3c6a-4e0d-9c6f-e8eb0f6409f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd4c6adbd09e8489b8349343477e85802'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:34:59.816368 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1409/2799] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:34:59 2022] DELETE /identity/v3/users/d4c6adbd09e8489b8349343477e85802 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:00.090191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.091740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:00.105378 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e63f825-cdf7-4908-a743-9001faac3b41 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.161858 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.162432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.163207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.163502 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] PATH_INFO: `/v3/projects/4e617d028c6d4448bc751413bf3c459d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.164670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e617d028c6d4448bc751413bf3c459d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.166564 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.196331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:00.214331 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e617d028c6d4448bc751413bf3c459d', 'request_id': 'req-23f4be61-295b-4d01-bece-b88107d1ac3f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:00.220780 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e617d028c6d4448bc751413bf3c459d', 'request_id': 'req-23f4be61-295b-4d01-bece-b88107d1ac3f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:00.228423 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e617d028c6d4448bc751413bf3c459d', 'request_id': 'req-23f4be61-295b-4d01-bece-b88107d1ac3f'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:00.238840 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] The token cache is being invalidate because project 4e617d028c6d4448bc751413bf3c459d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:00.239128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:00.239803 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-23f4be61-295b-4d01-bece-b88107d1ac3f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:00.241949 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1391/2800] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:00 2022] DELETE /identity/v3/projects/4e617d028c6d4448bc751413bf3c459d => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:00.246780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.247262 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.247363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.266243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:00.267407 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:00.324958 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1410/2801] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:00.333118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.333394 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.333661 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.353007 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:00.354173 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:00.361817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1392/2802] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:00.366456 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.376209 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215b2fa5-726d-4063-9bfe-8d28ca82576b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.380841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.381475 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.381754 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.382029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.382743 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:35:00.383045 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.384676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.395969 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1411/2803] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:35:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:35:00.400019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.409072 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90df9d9b-373f-405e-82fa-facf0771905d None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.414161 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.414691 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.414966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.415234 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.415946 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.417854 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.444826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1393/2804] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/projects => generated 381 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:00.448929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.458905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65eb436-c021-45e7-a101-a44888856c58 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.484827 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:00.516952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.517561 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.517840 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.518131 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.518809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.520734 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.540903 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1412/2805] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/users => generated 291 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:00.545047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.555676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ddbddb-0ce1-450a-a5fa-957008d8eec3 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.560220 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.560824 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.561105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.561381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.562078 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.563705 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.573957 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1394/2806] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:00.577953 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.588227 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-553352c0-cd12-4492-9051-4be0644d53e7 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.592925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.593525 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.593832 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.594103 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.594819 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.596428 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e28591a9-02b7-41bf-8b0c-ea4a5fdc5eae admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.606891 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1413/2807] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:00.610910 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.621351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6ba2a4-04d6-40b3-9efa-151ec0c50833 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.626080 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.626731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.627015 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.627288 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] PATH_INFO: `/v3/projects/e47a67d2cb524dfa948b4977b84ee681/users/72c35d3166984489b044e248b4bd2cae/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.642143 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e47a67d2cb524dfa948b4977b84ee681, user_id=72c35d3166984489b044e248b4bd2cae, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.644825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.657815 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1395/2808] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:00 2022] PUT /identity/v3/projects/e47a67d2cb524dfa948b4977b84ee681/users/72c35d3166984489b044e248b4bd2cae/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:00.668198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.668536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.668870 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.696849 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] MFA Rules not processed for user `72c35d3166984489b044e248b4bd2cae`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:00.698043 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:00.760251 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1414/2809] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:00.764859 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.825147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9751fd1a-4637-4217-b292-5447352f490c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.830049 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.830657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.830938 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.831214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.831918 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.833811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.860905 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1396/2810] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/projects => generated 375 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:00.864911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.876026 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45877ec9-6f63-4ce1-aba9-9045505d6bc5 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.931790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.932072 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.932419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.932714 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.933509 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.935810 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.956394 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1415/2811] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:00 2022] POST /identity/v3/users => generated 294 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:00.960554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:00.970622 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa452182-6e0b-4567-bb6b-6dc5670d3da6 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:00.975158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:00.976351 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:00.976624 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:00.976895 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:00.977582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:00.979312 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:00.989710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1397/2812] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:00 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 31 14:35:00.994503 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:01.004510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc3ffaa-dfbf-41da-adcf-e56ac9fb03c6 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:01.009168 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:01.009821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.010104 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.010385 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] PATH_INFO: `/v3/projects/0063cf369f354bcabb95ae63a305a872/users/d9c756ab436241a1a3fe934691b58305/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.025021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063cf369f354bcabb95ae63a305a872, user_id=d9c756ab436241a1a3fe934691b58305, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:01.027901 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:01.042009 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1416/2813] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:00 2022] PUT /identity/v3/projects/0063cf369f354bcabb95ae63a305a872/users/d9c756ab436241a1a3fe934691b58305/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:35:01.046145 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:01.056915 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577226e-15a3-446c-a101-092b92c37f53 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:01.076216 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:01.114657 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:01.115275 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.115592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.115870 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.117113 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:01.118707 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6473ec0c-54f6-45f0-9e08-d4799bba3ee8 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:01.128290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1398/2814] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:01 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:01.132628 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:01.138156 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:01.143698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06746e9-1471-40c9-9997-a17a2271c914 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:01.149504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:01.150199 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.150482 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.150761 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] PATH_INFO: `/v3/projects/0063cf369f354bcabb95ae63a305a872/users/d9c756ab436241a1a3fe934691b58305/roles/f459ad5c5a244da98c2cacf1a3be0871` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.152752 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0063cf369f354bcabb95ae63a305a872, user_id=d9c756ab436241a1a3fe934691b58305, role_id=f459ad5c5a244da98c2cacf1a3be0871)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:01.155413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:01.166273 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1417/2815] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:01 2022] PUT /identity/v3/projects/0063cf369f354bcabb95ae63a305a872/users/d9c756ab436241a1a3fe934691b58305/roles/f459ad5c5a244da98c2cacf1a3be0871 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:01.232832 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.233379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.233379 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.261328 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] MFA Rules not processed for user `d9c756ab436241a1a3fe934691b58305`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:01.262530 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:01.328130 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1399/2816] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:01 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:01.339781 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:01.341516 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:01.397305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d4e789a-2e7c-42ed-b17a-c4b5cec878e0 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:01.401356 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:01.401956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.402234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.402500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.411090 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:01.412522 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:01.422549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1418/2817] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:01.497987 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:01.507636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ddd95c-4bab-481a-8a24-d22c26548109 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:01.511286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:01.511966 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:01.512247 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:01.512524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:01.567742 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:01.569171 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8442317-3763-4735-9952-e08b342dc4b2 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:01.578962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1400/2818] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:01 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:10.381101 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:10.442084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb2be04-ae45-41fe-b78c-fe0e1dd6c1b4 service cinder] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:10.446723 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:10.447354 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:10.447665 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:10.447942 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] PATH_INFO: `/v3/users/72c35d3166984489b044e248b4bd2cae` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:10.449109 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72c35d3166984489b044e248b4bd2cae)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:10.451050 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:10.505438 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72c35d3166984489b044e248b4bd2cae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:10.516952 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72c35d3166984489b044e248b4bd2cae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:10.517629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72c35d3166984489b044e248b4bd2cae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:10.525612 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72c35d3166984489b044e248b4bd2cae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:10.526269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72c35d3166984489b044e248b4bd2cae'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:10.538725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1419/2819] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:35:10 2022] DELETE /identity/v3/users/72c35d3166984489b044e248b4bd2cae => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:10.543227 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:10.543741 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:10.543849 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:10.562513 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:10.563752 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:10.612200 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=100 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:10.633062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1401/2820] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:10.642944 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:10.644366 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:10.658524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:10.703056 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb741b69-be47-4768-9130-3def3eaed3a7 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:10.706709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:10.707321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:10.707636 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:10.707914 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:10.717562 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:10.719003 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:10.728379 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1420/2821] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:11.007568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.009012 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:11.020955 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8b2aec-a97f-4e75-bcca-5e07a1a60afe None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.026138 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.026958 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.027241 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.027684 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] PATH_INFO: `/v3/projects/e47a67d2cb524dfa948b4977b84ee681` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.028866 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e47a67d2cb524dfa948b4977b84ee681)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.030787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:11.078213 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e47a67d2cb524dfa948b4977b84ee681', 'request_id': 'req-24f75261-b8cd-44ef-a75d-98f9ed5eb911'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.084198 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e47a67d2cb524dfa948b4977b84ee681', 'request_id': 'req-24f75261-b8cd-44ef-a75d-98f9ed5eb911'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.091636 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e47a67d2cb524dfa948b4977b84ee681', 'request_id': 'req-24f75261-b8cd-44ef-a75d-98f9ed5eb911'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.101779 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] The token cache is being invalidate because project e47a67d2cb524dfa948b4977b84ee681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:11.102062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.102678 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.104207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1402/2822] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:11 2022] DELETE /identity/v3/projects/e47a67d2cb524dfa948b4977b84ee681 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:11.108279 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.118271 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f02ec3-014d-47b1-a4a5-cb1d197e5099 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.173864 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.174502 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.174780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.175054 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] PATH_INFO: `/v3/users/d9c756ab436241a1a3fe934691b58305` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.176247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9c756ab436241a1a3fe934691b58305)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.178192 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:11.222457 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9c756ab436241a1a3fe934691b58305'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.233904 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9c756ab436241a1a3fe934691b58305'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.234548 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9c756ab436241a1a3fe934691b58305'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.240888 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9c756ab436241a1a3fe934691b58305'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.242064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-726911d4-cd72-4156-81c8-2ceec97b3385 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9c756ab436241a1a3fe934691b58305'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.254247 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1421/2823] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:35:11 2022] DELETE /identity/v3/users/d9c756ab436241a1a3fe934691b58305 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:11.529976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.531538 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:11.545185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75261-b8cd-44ef-a75d-98f9ed5eb911 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.608036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.609043 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.609337 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.609612 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] PATH_INFO: `/v3/projects/0063cf369f354bcabb95ae63a305a872` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.610825 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0063cf369f354bcabb95ae63a305a872)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.612757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:11.672808 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0063cf369f354bcabb95ae63a305a872', 'request_id': 'req-9e51beeb-6410-4064-8f3b-26002108a4c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.679381 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0063cf369f354bcabb95ae63a305a872', 'request_id': 'req-9e51beeb-6410-4064-8f3b-26002108a4c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.686896 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0063cf369f354bcabb95ae63a305a872', 'request_id': 'req-9e51beeb-6410-4064-8f3b-26002108a4c5'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.693191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:11.698925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] The token cache is being invalidate because project 0063cf369f354bcabb95ae63a305a872 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:11.699202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.699952 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-9e51beeb-6410-4064-8f3b-26002108a4c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:11.702361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1403/2824] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:11 2022] DELETE /identity/v3/projects/0063cf369f354bcabb95ae63a305a872 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:11.708500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.709155 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.709442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.728780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:11.729977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:11.758836 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:11.789894 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1422/2825] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:11.798059 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.798365 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.798676 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.818380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:11.820297 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:11.828377 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1404/2826] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:11.832704 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.842346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccd9408-10b3-43d6-913f-9286fe9c7d5a None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.847027 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.847676 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.847961 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.848236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.848929 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:35:11.849236 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.850823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:11.862128 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1423/2827] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:35:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:35:11.866120 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.876097 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19b4420-a6a0-4782-9d54-08db6d8247da None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.880664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.881257 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.881535 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.881807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.882468 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.884982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:11.911685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1405/2828] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:11 2022] POST /identity/v3/projects => generated 373 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:11.915478 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:11.925510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4b38f-2aca-4258-aa04-78f71fe1fa4a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:11.984363 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:11.984978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:11.985259 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:11.985537 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:11.986208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:11.988152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:12.009722 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1424/2829] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:11 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:12.013888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:12.024174 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5c6450-4b14-46d3-b925-f45f3a0cc5ac admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:12.028796 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:12.029403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:12.029685 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:12.029962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:12.030660 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:12.032305 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:12.042484 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1406/2830] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:12.046459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:12.056890 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e6203-af05-4b02-bd5c-b00448f14af5 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:12.061504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:12.062108 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:12.062391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:12.062668 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:12.063362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:12.064965 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5f4e66-dac0-4922-ae4f-dd117724ea98 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:12.075373 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1425/2831] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:12 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:12.079759 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:12.089536 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dec827-81ad-4a04-afef-bc2b9a47443f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:12.094085 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:12.094735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:12.095041 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:12.095318 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] PATH_INFO: `/v3/projects/374e273c2a8f45a389c344e39cef6215/users/1df24b7131f743a8ac517d2adc623448/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:12.110230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=374e273c2a8f45a389c344e39cef6215, user_id=1df24b7131f743a8ac517d2adc623448, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:12.113032 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:12.134744 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1407/2832] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:12 2022] PUT /identity/v3/projects/374e273c2a8f45a389c344e39cef6215/users/1df24b7131f743a8ac517d2adc623448/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:12.136180 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:12.136467 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:12.136739 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:12.163390 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] MFA Rules not processed for user `1df24b7131f743a8ac517d2adc623448`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:12.165115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:12.228433 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1426/2833] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:12 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:12.239664 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:12.241452 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:12.319107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75a9a94-312e-4bff-b619-221a02ac6dbe admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:12.322799 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '42b59618e59540d1bcd5c2397b004422', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:12.323403 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:12.323710 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:12.323976 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:12.332214 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:12.333655 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103556d0-d942-4d59-8bc8-204b6ea0b5a2 service glance] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:12.342158 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1408/2834] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:12 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:22.225053 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=84,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:22.338273 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=63,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:27.841614 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:27.906419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7acb6-4f6e-4804-899f-abd097a0f92b None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:27.910979 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:27.911655 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:27.911938 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:27.912216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] PATH_INFO: `/v3/users/1df24b7131f743a8ac517d2adc623448` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:27.913375 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1df24b7131f743a8ac517d2adc623448)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:27.915287 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:27.966506 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1df24b7131f743a8ac517d2adc623448'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:27.978139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1df24b7131f743a8ac517d2adc623448'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:27.979305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1df24b7131f743a8ac517d2adc623448'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:27.986765 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1df24b7131f743a8ac517d2adc623448'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:27.987411 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1df24b7131f743a8ac517d2adc623448'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.000335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1427/2835] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:35:27 2022] DELETE /identity/v3/users/1df24b7131f743a8ac517d2adc623448 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:28.005061 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.005620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.005620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.024062 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:28.025254 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:28.084823 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1409/2836] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:28.094500 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.095991 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:28.161245 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8817b7-61b7-4a6d-9eda-1b879d49fe41 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.164939 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.165560 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.166461 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.166737 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.175824 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.177253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b610c7-f18f-4fc0-ade8-44b3ecb7aff1 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.186613 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1428/2837] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:35:28.457408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.458848 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:28.471346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ef8760-91ff-409c-b05c-6dd061083878 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.475899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.476575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.476855 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.477128 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] PATH_INFO: `/v3/projects/374e273c2a8f45a389c344e39cef6215` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.478940 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] RBAC: Authorizing `identity:delete_project(project_id=374e273c2a8f45a389c344e39cef6215)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.480867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.527207 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '374e273c2a8f45a389c344e39cef6215', 'request_id': 'req-094e86dc-492a-4a2e-b2b5-97ba6f37c871'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.533192 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '374e273c2a8f45a389c344e39cef6215', 'request_id': 'req-094e86dc-492a-4a2e-b2b5-97ba6f37c871'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.540549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '374e273c2a8f45a389c344e39cef6215', 'request_id': 'req-094e86dc-492a-4a2e-b2b5-97ba6f37c871'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.550860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] The token cache is being invalidate because project 374e273c2a8f45a389c344e39cef6215 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:28.551147 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.551788 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-094e86dc-492a-4a2e-b2b5-97ba6f37c871 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:28.553321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1410/2838] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:28 2022] DELETE /identity/v3/projects/374e273c2a8f45a389c344e39cef6215 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:28.558221 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.558513 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.558795 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.578371 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:28.579622 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:28.596902 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:28.641315 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1429/2839] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:28.648982 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.649269 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.649545 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.676045 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:28.677873 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:28.689242 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1411/2840] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:28.693597 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.704208 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a63d50-3aae-4517-851b-2af88f117f49 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.708686 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.709291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.709568 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.709841 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.710526 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:35:28.710825 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.712435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.724805 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1430/2841] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:35:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:35:28.729107 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.739191 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b820e3d0-fe79-4aa0-a6d0-69f83213219a None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.743805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.744415 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.744692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.744963 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.745617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.747620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.774296 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1412/2842] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/projects => generated 359 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:28.779215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.789233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54af3702-b3e6-4f5d-8928-33f8777ada8a admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.844355 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.844993 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.845233 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.845504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.846157 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.848095 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.868780 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1431/2843] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/users => generated 280 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:35:28.873425 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.883504 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5502f4-5839-471e-9ac2-415a78a9e3e7 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.888563 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.889165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.889443 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.889714 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.890432 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.892112 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.901842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1413/2844] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:28.905732 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.916042 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b839794-9afa-4a60-8de9-c5ceb8ac6481 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.920910 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.921527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.921809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.922082 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.922801 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.924442 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c439cb-19d2-4b45-9b57-776aef2b5fbd admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.935764 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1432/2845] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:28.940514 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:28.951464 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f21194-1ba6-4780-b3dc-3f5e68929017 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:28.956342 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:28.956998 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.957286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.957565 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] PATH_INFO: `/v3/projects/816b4b0e1b8c492d8459161f5e10088d/users/ba234a18376a4b2e8ea8e8f5354edb8b/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:28.971757 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] RBAC: Authorizing `identity:create_grant(project_id=816b4b0e1b8c492d8459161f5e10088d, user_id=ba234a18376a4b2e8ea8e8f5354edb8b, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:28.974395 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:28.987185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1414/2846] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:28 2022] PUT /identity/v3/projects/816b4b0e1b8c492d8459161f5e10088d/users/ba234a18376a4b2e8ea8e8f5354edb8b/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:28.998018 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:28.998305 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:28.998582 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.027276 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] MFA Rules not processed for user `ba234a18376a4b2e8ea8e8f5354edb8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:29.028549 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.091113 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1433/2847] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:28 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:29.123652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:29.125361 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.147589 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:29.183317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c4e8f-9897-4d00-89c6-cd64cebcb492 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:29.186964 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:29.187616 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.187894 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.188169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.196346 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:29.198338 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8d8a37-1cf4-4b9f-9c4d-4098ec70fc28 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:29.207412 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1415/2848] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:29.222342 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:29.241238 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:29.284621 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def884f4-4400-4cc2-989d-aa9a8e88c307 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:29.289162 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:29.289790 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.290066 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.290337 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] PATH_INFO: `/v3/users/ba234a18376a4b2e8ea8e8f5354edb8b` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.291999 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba234a18376a4b2e8ea8e8f5354edb8b)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:29.294164 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:29.341261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba234a18376a4b2e8ea8e8f5354edb8b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.352700 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba234a18376a4b2e8ea8e8f5354edb8b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.353856 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba234a18376a4b2e8ea8e8f5354edb8b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.360978 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba234a18376a4b2e8ea8e8f5354edb8b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.361609 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba234a18376a4b2e8ea8e8f5354edb8b'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.374047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1434/2849] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:35:29 2022] DELETE /identity/v3/users/ba234a18376a4b2e8ea8e8f5354edb8b => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:29.378640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.379017 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.379302 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.397756 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:29.398875 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.455435 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1416/2850] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:29.465006 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:29.466439 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.523357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb70ade-164d-4fe7-b620-273512160fea admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:29.526959 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:29.527595 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.527872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.528139 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.536898 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:29.538306 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d624edb-eee4-4e9b-94e6-700cced1c459 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:29.547947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1435/2851] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:29.816294 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:29.817794 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.832261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0279e934-81bb-4591-b3e6-5307fedd8e04 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:29.844479 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:29.845203 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.845488 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.845764 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] PATH_INFO: `/v3/projects/816b4b0e1b8c492d8459161f5e10088d` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.846986 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] RBAC: Authorizing `identity:delete_project(project_id=816b4b0e1b8c492d8459161f5e10088d)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:29.848925 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:29.894582 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '816b4b0e1b8c492d8459161f5e10088d', 'request_id': 'req-ff9b004e-7741-4893-9c1c-c894eb9ce220'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.901079 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '816b4b0e1b8c492d8459161f5e10088d', 'request_id': 'req-ff9b004e-7741-4893-9c1c-c894eb9ce220'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.908380 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '816b4b0e1b8c492d8459161f5e10088d', 'request_id': 'req-ff9b004e-7741-4893-9c1c-c894eb9ce220'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.919445 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] The token cache is being invalidate because project 816b4b0e1b8c492d8459161f5e10088d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:29.919740 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.920364 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-ff9b004e-7741-4893-9c1c-c894eb9ce220 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:29.921885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1417/2852] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:29 2022] DELETE /identity/v3/projects/816b4b0e1b8c492d8459161f5e10088d => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:29.926945 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:29.927505 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:29.927630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:29.948577 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:29.949777 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:29.982963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:30.010641 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1436/2853] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:30.019942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.020231 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.020509 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.040405 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:30.041600 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:30.049317 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1418/2854] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:30.053587 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.063203 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-236b890d-999f-46a4-88ed-f427f52e3ad0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.067756 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.068365 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.068645 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.068920 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.069617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:35:30.069926 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.071543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.083867 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1437/2855] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:35:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:35:30.088047 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.097620 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd3ce7-c912-49b7-a5d5-89e102c8bdd3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.102268 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.102879 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.103137 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.103409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.104105 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.106025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.134339 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1419/2856] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:30 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 31 14:35:30.139234 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.149350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adbdb87f-f329-4dc1-b9ff-1a4b7d5f5633 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.203501 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.204044 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.204319 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.204589 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.205263 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.207152 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.228320 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1438/2857] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:30 2022] POST /identity/v3/users => generated 288 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:30.232417 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.242549 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61949b7e-5c36-48d0-9f93-e4752268dc52 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.247148 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.247798 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.248076 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.248348 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.249044 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.250623 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.260787 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1420/2858] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:30.264504 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.274261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94c5f1-adf8-4f86-a21a-8ab5ba6a4940 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.279492 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.280097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.280381 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.280654 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.281344 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.282911 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8243af2d-ba1e-4320-b453-93261fc250d2 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.293264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1439/2859] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:30 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:30.297230 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.306965 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f573ec-6ff1-4cfa-ab3b-60a4045d9860 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.311543 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.312196 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.312474 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.312749 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] PATH_INFO: `/v3/projects/376c3e540d704e8b841899620b28a2e1/users/3e9c86414be5442a8b66cdcda84edeec/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.314992 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:30.330023 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] RBAC: Authorizing `identity:create_grant(project_id=376c3e540d704e8b841899620b28a2e1, user_id=3e9c86414be5442a8b66cdcda84edeec, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.332698 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.345512 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1421/2860] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:30 2022] PUT /identity/v3/projects/376c3e540d704e8b841899620b28a2e1/users/3e9c86414be5442a8b66cdcda84edeec/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:30.355625 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.355916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.356218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.383115 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] MFA Rules not processed for user `3e9c86414be5442a8b66cdcda84edeec`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:30.384346 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:30.447916 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1440/2861] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:30 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:30.458735 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:30.460358 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:30.517236 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30990bc7-43d1-4c7e-b74a-5283920a0b3c admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:30.520842 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:30.521453 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:30.521760 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:30.522031 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:30.530201 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:30.531652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62268025-920b-4646-9722-5980cc0823b7 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:30.540494 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1422/2862] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:30 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:40.444335 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=81,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:40.536409 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=33,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:42.424169 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:42.485431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c5f0a-4b70-4c26-9b73-5742e52c2cd0 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:42.489899 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:42.490531 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:42.490811 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:42.491084 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] PATH_INFO: `/v3/users/3e9c86414be5442a8b66cdcda84edeec` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:42.492254 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e9c86414be5442a8b66cdcda84edeec)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:42.494173 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:42.538820 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e9c86414be5442a8b66cdcda84edeec'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:42.551530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e9c86414be5442a8b66cdcda84edeec'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:42.552191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e9c86414be5442a8b66cdcda84edeec'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:42.559150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e9c86414be5442a8b66cdcda84edeec'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:42.559789 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e9c86414be5442a8b66cdcda84edeec'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:42.571981 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1441/2863] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:35:42 2022] DELETE /identity/v3/users/3e9c86414be5442a8b66cdcda84edeec => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 31 14:35:42.576458 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:42.576934 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:42.577039 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:42.596154 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:42.597345 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:42.654718 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1423/2864] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:42.664269 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:42.665671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:42.743150 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d386181b-b101-443c-9ce1-4934d2b9aeea admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:42.748429 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:42.749063 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:42.749339 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:42.749618 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:42.763976 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:42.765444 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e65050-ed87-4880-a5e5-b9b70d06dc7c service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:42.774987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1442/2865] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:43.041221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.043586 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:43.065519 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5e4d17-b851-4e6d-9d20-4db87dfd4999 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.073653 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.074734 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.075807 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.076310 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] PATH_INFO: `/v3/projects/376c3e540d704e8b841899620b28a2e1` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.078383 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] RBAC: Authorizing `identity:delete_project(project_id=376c3e540d704e8b841899620b28a2e1)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.081103 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.133321 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '376c3e540d704e8b841899620b28a2e1', 'request_id': 'req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:43.140027 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '376c3e540d704e8b841899620b28a2e1', 'request_id': 'req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:43.147298 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '376c3e540d704e8b841899620b28a2e1', 'request_id': 'req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:43.157834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] The token cache is being invalidate because project 376c3e540d704e8b841899620b28a2e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80121) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:35:43.158149 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:43.158762 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-f7d2e2e1-390e-4ebb-aac9-f259ed6d1498 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:35:43.160386 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1424/2866] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:35:43 2022] DELETE /identity/v3/projects/376c3e540d704e8b841899620b28a2e1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:43.165237 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.165527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.165829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.185473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:43.186679 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:43.203850 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:43.246266 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1443/2867] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:43.254239 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.254526 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.254802 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.274887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:43.276110 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:43.283860 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1425/2868] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:43.287988 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.297440 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c941d2ad-708f-4a5f-8d4b-c666718b1a29 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.301977 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.302580 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.302857 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.303126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.303854 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80120) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 31 14:35:43.304185 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.305770 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.317524 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1444/2869] 10.222.0.40 () {58 vars in 1153 bytes} [Sun Jul 31 14:35:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 31 14:35:43.323816 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.333100 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db35d064-db14-4ec9-a350-73581a90c136 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.337811 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.338410 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.338689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.338962 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] PATH_INFO: `/v3/projects` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.339652 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.341524 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.368281 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1426/2870] 10.222.0.40 () {60 vars in 1153 bytes} [Sun Jul 31 14:35:43 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:43.372855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.382698 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca75a12-bb51-4a15-a808-3a75ebb64ba7 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.438215 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.438838 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.439021 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.439291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] PATH_INFO: `/v3/users` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.440036 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.441956 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.462596 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1445/2871] 10.222.0.40 () {60 vars in 1144 bytes} [Sun Jul 31 14:35:43 2022] POST /identity/v3/users => generated 292 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 31 14:35:43.466743 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.476888 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19dd3a0-2263-4f48-b885-2b192d1a0478 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.482132 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.482739 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.483016 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.483285 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.484022 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.485668 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.496102 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1427/2872] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:43.500172 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.509925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6696f7cc-627d-496f-ab42-7bca4143c016 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.514473 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.515073 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.515352 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.515658 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.516350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.517925 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d87e2b7-52f4-4d82-9869-9a454a2395c9 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.528278 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1446/2873] 10.222.0.40 () {58 vars in 1122 bytes} [Sun Jul 31 14:35:43 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 31 14:35:43.532246 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.542354 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c7a7a84-a57f-4001-98c9-75180d460c25 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.546797 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.547467 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] REQUEST_METHOD: `PUT` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.547747 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.548019 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] PATH_INFO: `/v3/projects/641f0c2e10be46218a8cb78b8ca9b62c/users/e0d7c887fc7b4987ad0bb94a5a99de5c/roles/dcc526de6b3241fa9ae5f10eb055621a` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.562609 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] RBAC: Authorizing `identity:create_grant(project_id=641f0c2e10be46218a8cb78b8ca9b62c, user_id=e0d7c887fc7b4987ad0bb94a5a99de5c, role_id=dcc526de6b3241fa9ae5f10eb055621a)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.565290 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.578169 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1428/2874] 10.222.0.40 () {60 vars in 1483 bytes} [Sun Jul 31 14:35:43 2022] PUT /identity/v3/projects/641f0c2e10be46218a8cb78b8ca9b62c/users/e0d7c887fc7b4987ad0bb94a5a99de5c/roles/dcc526de6b3241fa9ae5f10eb055621a => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:35:43.582963 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c041b-fbd8-42f8-9ebb-34f3baa3057f None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.583249 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c041b-fbd8-42f8-9ebb-34f3baa3057f None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.583565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13c041b-fbd8-42f8-9ebb-34f3baa3057f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.601709 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-d13c041b-fbd8-42f8-9ebb-34f3baa3057f None None] MFA Rules not processed for user `da5b1c90e892464380eacee9a29f0449`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:43.602930 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-d13c041b-fbd8-42f8-9ebb-34f3baa3057f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:43.658970 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1447/2875] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:43.669131 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:43.670805 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:43.696835 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:43.727177 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36fb5f2-fb20-4e5c-a617-a8f547f0473f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:43.731261 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:43.731907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:43.732182 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:43.732449 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:43.741096 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:43.742534 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:43.752295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1429/2876] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:35:44.361639 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55945d-bc97-4b18-91cb-4e693888eb06 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:44.361931 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55945d-bc97-4b18-91cb-4e693888eb06 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:44.362211 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec55945d-bc97-4b18-91cb-4e693888eb06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:44.390198 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-ec55945d-bc97-4b18-91cb-4e693888eb06 None None] MFA Rules not processed for user `e0d7c887fc7b4987ad0bb94a5a99de5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:44.391565 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ec55945d-bc97-4b18-91cb-4e693888eb06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:44.408550 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:44.466159 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1448/2877] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:44 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:44.479899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:44.481387 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:44.541036 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f58cbaa-023c-4345-b07e-8816c84019fe service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:44.544683 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5bd24147c8c44ff7a9800bc406d51953', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:44.545300 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:44.545575 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:44.545847 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:44.554163 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:44.555617 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd325015-43fc-4a09-bc69-7f80ca4870f8 service cinder] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:44.565520 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1430/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:44 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:45.092323 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] REQUEST_METHOD: `POST` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:45.092617 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:45.092915 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:45.111809 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.auth.core [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] MFA Rules not processed for user `0bbe35ad557841c581e3b1a4aa272f38`. Rule list: `[]` (Enabled: `True`). {{(pid=80120) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:45.113004 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:45.167829 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1449/2879] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:45.923669 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:45.924167 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:45.924292 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:45.944924 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] MFA Rules not processed for user `e0d7c887fc7b4987ad0bb94a5a99de5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:35:45.946264 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:45.953591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1431/2880] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:35:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:35:46.207340 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:35:46.217646 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e7e2bb-0dad-4aff-a933-a44efeb25b91 None None] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:35:46.221431 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:35:46.222137 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:35:46.222484 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:35:46.222823 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:35:46.224855 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:35:46.235725 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:35:46.237166 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:35:46.245536 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1450/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:35:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:35:55.939517 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=46 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:35:56.242435 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=50 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:36:04.728450 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:36:04.730413 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:36:04.808640 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ece07e-cf85-4830-a3e4-9b4ee5603ade None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:36:04.814008 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd04ab058ac79408f80cc485082c6591c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'service', 'admin', 'ResellerAdmin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:36:04.814408 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:36:04.814688 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:36:04.815025 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:36:04.816871 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:36:04.830074 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:36:04.831729 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252d42f8-2225-4308-b403-671997b8ab74 service nova] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:36:04.844010 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1432/2882] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:36:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:36:14.838313 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=34 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:36:40.227195 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:36:40.228657 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:36:40.299291 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441c5017-df93-491f-bc70-c1bd3c3a39f3 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:36:40.304462 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:36:40.305029 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:36:40.305312 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:36:40.305588 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:36:40.307357 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:36:40.319191 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:36:40.320792 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:36:40.331559 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1451/2883] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:36:40 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:36:49.154950 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] REQUEST_METHOD: `POST` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:36:49.155469 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:36:49.155580 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:36:49.175602 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.auth.core [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] MFA Rules not processed for user `e0d7c887fc7b4987ad0bb94a5a99de5c`. Rule list: `[]` (Enabled: `True`). {{(pid=80121) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 31 14:36:49.176817 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:36:49.183591 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1433/2884] 10.222.0.40 () {58 vars in 958 bytes} [Sun Jul 31 14:36:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 31 14:36:50.326543 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=32 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:36:59.169689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=3 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:37:45.362796 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:45.373220 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cd5dbf7-84fd-415f-8557-2bbc4a736f1c service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:45.377419 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:45.378047 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:45.378327 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:45.378630 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:45.380384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:45.391196 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:45.393362 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:45.402218 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1452/2885] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:37:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:37:47.335593 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:47.348197 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af26cc0-de89-490a-9672-965e1a1ccee3 None None] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:47.352885 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:47.353531 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:47.353810 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:47.354084 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] PATH_INFO: `/v3/users/e0d7c887fc7b4987ad0bb94a5a99de5c` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:47.355271 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e0d7c887fc7b4987ad0bb94a5a99de5c)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:47.357202 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:47.402584 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e0d7c887fc7b4987ad0bb94a5a99de5c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.414040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e0d7c887fc7b4987ad0bb94a5a99de5c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.414692 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e0d7c887fc7b4987ad0bb94a5a99de5c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.415284 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e0d7c887fc7b4987ad0bb94a5a99de5c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.421999 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e0d7c887fc7b4987ad0bb94a5a99de5c'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.435172 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1434/2886] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:37:47 2022] DELETE /identity/v3/users/e0d7c887fc7b4987ad0bb94a5a99de5c => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:37:47.707881 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:47.720321 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4140e9-0c36-4986-ae9c-fa86745bbe0e service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:47.800252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:47.800905 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:47.801190 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:47.801459 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] PATH_INFO: `/v3/projects/641f0c2e10be46218a8cb78b8ca9b62c` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:47.802611 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] RBAC: Authorizing `identity:delete_project(project_id=641f0c2e10be46218a8cb78b8ca9b62c)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:47.804578 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:47.853671 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '641f0c2e10be46218a8cb78b8ca9b62c', 'request_id': 'req-bae29907-bafb-49a6-b294-505f28a70427'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.862253 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '641f0c2e10be46218a8cb78b8ca9b62c', 'request_id': 'req-bae29907-bafb-49a6-b294-505f28a70427'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.868437 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '641f0c2e10be46218a8cb78b8ca9b62c', 'request_id': 'req-bae29907-bafb-49a6-b294-505f28a70427'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.879158 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] The token cache is being invalidate because project 641f0c2e10be46218a8cb78b8ca9b62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:37:47.879464 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.880097 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:37:47.881586 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1453/2887] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:37:47 2022] DELETE /identity/v3/projects/641f0c2e10be46218a8cb78b8ca9b62c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:37:53.436108 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:53.438363 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:53.504592 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee3b338-87e8-4bc8-809c-a52d23085248 admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:53.553826 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:53.554440 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:53.554715 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:53.554985 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:53.557099 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:53.602603 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:37:53.633706 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:53.635185 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:53.647040 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1435/2888] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:37:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 31 14:37:54.831738 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:54.833252 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:54.845463 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae29907-bafb-49a6-b294-505f28a70427 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:54.849629 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:54.850243 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:54.850557 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:54.850835 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:54.852644 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:54.867309 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:54.869064 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:54.880100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1454/2889] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:37:54 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:37:57.878385 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:37:57.887594 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ccc43-3ad6-4e3b-ae92-35443e7dfc36 service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:37:57.891118 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'a3c98dfd1f934a98b26889cda2d5c762', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:37:57.891758 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:37:57.892035 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:37:57.892304 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:37:57.893997 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:37:57.958834 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:37:57.960218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:37:57.969913 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1436/2890] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:37:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:38:01.072039 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:38:01.073545 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:38:01.178261 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b4ad80-2ad4-4103-802b-cb1177cb07b2 service neutron] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:38:01.183499 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:38:01.184100 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] REQUEST_METHOD: `GET` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:38:01.184384 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:38:01.184660 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:38:01.186413 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:38:01.195126 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:38:01.200527 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:38:01.202927 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:38:01.214107 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1455/2891] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:38:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:38:03.991828 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:38:03.994554 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:38:04.010719 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b32195-e654-48f7-adb1-dd0fa763d2aa service neutron] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:38:04.015528 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:38:04.016180 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:38:04.016456 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:38:04.016727 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] PATH_INFO: `/v3/users/0bbe35ad557841c581e3b1a4aa272f38` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:38:04.017892 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bbe35ad557841c581e3b1a4aa272f38)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:38:04.019903 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:38:04.069942 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bbe35ad557841c581e3b1a4aa272f38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.082397 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bbe35ad557841c581e3b1a4aa272f38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.083068 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bbe35ad557841c581e3b1a4aa272f38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.083731 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bbe35ad557841c581e3b1a4aa272f38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.090454 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bbe35ad557841c581e3b1a4aa272f38'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.103286 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1437/2892] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:38:03 2022] DELETE /identity/v3/users/0bbe35ad557841c581e3b1a4aa272f38 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:38:04.280872 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:38:04.297510 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ff1619-ee4d-4ff7-b48c-435d2c7b6672 service placement] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:38:04.359170 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:38:04.359913 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:38:04.360119 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:38:04.360391 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] PATH_INFO: `/v3/projects/8561e7f1f3d040e28d5084ebf937e198` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:38:04.361530 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8561e7f1f3d040e28d5084ebf937e198)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:38:04.363538 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:38:04.409136 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8561e7f1f3d040e28d5084ebf937e198', 'request_id': 'req-74beeb77-b13b-43cd-b567-88862ddc52c8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.417122 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8561e7f1f3d040e28d5084ebf937e198', 'request_id': 'req-74beeb77-b13b-43cd-b567-88862ddc52c8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.423298 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8561e7f1f3d040e28d5084ebf937e198', 'request_id': 'req-74beeb77-b13b-43cd-b567-88862ddc52c8'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.433696 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] The token cache is being invalidate because project 8561e7f1f3d040e28d5084ebf937e198 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:38:04.433987 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.434626 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.436216 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1456/2893] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:38:04 2022] DELETE /identity/v3/projects/8561e7f1f3d040e28d5084ebf937e198 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:38:04.440209 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:38:04.441670 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:38:04.454686 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f65aff-23f0-4138-bcd8-795c4dd902cc admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:38:04.487465 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:38:04.513900 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:38:04.514541 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:38:04.514822 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:38:04.515098 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] PATH_INFO: `/v3/users/c499de2c372642ff91e409f8c2882a74` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:38:04.516295 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c499de2c372642ff91e409f8c2882a74)` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:38:04.518218 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:38:04.564165 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c499de2c372642ff91e409f8c2882a74'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.576252 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c499de2c372642ff91e409f8c2882a74'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.576907 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c499de2c372642ff91e409f8c2882a74'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.577508 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c499de2c372642ff91e409f8c2882a74'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.585005 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.notifications [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c499de2c372642ff91e409f8c2882a74'} {{(pid=80121) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.597899 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1438/2894] 10.222.0.40 () {58 vars in 1224 bytes} [Sun Jul 31 14:38:04 2022] DELETE /identity/v3/users/c499de2c372642ff91e409f8c2882a74 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:38:04.769300 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Authenticating user token {{(pid=80120) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:38:04.771332 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.fernet_utils [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80120) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:38:04.783821 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74beeb77-b13b-43cd-b567-88862ddc52c8 admin admin] Validating token access rules against request {{(pid=80120) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:38:04.838998 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'da5b1c90e892464380eacee9a29f0449', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0e21ca24dfa94435a41c5ab2bd838cb9', 'project_domain_id': 'default', 'roles': ['reader', 'member', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80120) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:38:04.839669 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:38:04.839947 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:38:04.840217 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] PATH_INFO: `/v3/projects/7e920c5683554b9a830c8d4ec011059f` {{(pid=80120) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:38:04.841350 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e920c5683554b9a830c8d4ec011059f)` {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:38:04.843242 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] RBAC: Authorization granted {{(pid=80120) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:38:04.873264 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:38:04.890455 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e920c5683554b9a830c8d4ec011059f', 'request_id': 'req-2af712f3-df1a-4c83-ac86-820dd1dbade0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.897683 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e920c5683554b9a830c8d4ec011059f', 'request_id': 'req-2af712f3-df1a-4c83-ac86-820dd1dbade0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.903816 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e920c5683554b9a830c8d4ec011059f', 'request_id': 'req-2af712f3-df1a-4c83-ac86-820dd1dbade0'} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.914417 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] The token cache is being invalidate because project 7e920c5683554b9a830c8d4ec011059f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80120) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 31 14:38:04.914697 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.915313 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG keystone.notifications [None req-2af712f3-df1a-4c83-ac86-820dd1dbade0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80120) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 31 14:38:04.917059 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: [pid: 80120|app: 0|req: 1457/2895] 10.222.0.40 () {58 vars in 1233 bytes} [Sun Jul 31 14:38:04 2022] DELETE /identity/v3/projects/7e920c5683554b9a830c8d4ec011059f => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 31 14:38:14.594486 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=37,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:38:14.912099 dv-cinder-849846-4-iscsi devstack@keystone.service[80120]: DEBUG dbcounter [-] [80120] Writing DB stats keystone:DELETE=6,keystone:SELECT=14,keystone:INSERT=1 {{(pid=80120) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 31 14:41:59.752867 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Authenticating user token {{(pid=80121) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 31 14:41:59.754253 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:41:59.891221 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b19d744-6579-4c33-87be-83930b099d8f admin admin] Validating token access rules against request {{(pid=80121) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 31 14:41:59.948075 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'dee88b8fbb6645f6a8df6a26c11941af', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'f9e2337dff494adbaf88dad516c3d98d', 'project_domain_id': 'default', 'roles': ['reader', 'service', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80121) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 31 14:41:59.948689 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] REQUEST_METHOD: `GET` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 31 14:41:59.948968 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] SCRIPT_NAME: `/identity` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 31 14:41:59.949240 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80121) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 31 14:41:59.950996 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.fernet_utils [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80121) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 31 14:41:59.962887 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 31 14:41:59.964568 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b65cd3-51ca-4145-8683-224be3c85a45 service placement] RBAC: Authorization granted {{(pid=80121) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 31 14:41:59.974784 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: [pid: 80121|app: 0|req: 1439/2896] 10.222.0.40 () {60 vars in 1365 bytes} [Sun Jul 31 14:41:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 31 14:42:09.970427 dv-cinder-849846-4-iscsi devstack@keystone.service[80121]: DEBUG dbcounter [-] [80121] Writing DB stats keystone:SELECT=89 {{(pid=80121) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}